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Throughout  the  text  of  tins  paper.  reference  is  made  lo  volume-  I  through  V.  These  volumes  have  liecn 
published  as  -eparale  leehnieal  papers  identified  as  follows: 

Volume  1 

Baum.  D.IL.  Modriek.  J.A..  X  Hollingsworth.  S.IL  /  cam  Iriiumif :  fur  run  niantl  unit  coatud  wstt-m-: 
's/trlus.  AFHHL-1 K-82- . .  \\  righl-l’atlerson  \f  B.  Oil :  Logistic-  and  I  erhm*  a  I  I’ raining  I  h\  ision.  \ir 
f  orce  Human  Resources  Lahoralorv  .  \pril  ll)82. 

Volume  II 

Modriek.  J.A..  Baum.  D.R..  ,S  Hollingsworth.  ,s. H .  Ictini  * riimin e  far  coiitmaad  tint I  ttmlroi  systems: 
Recommendations  fa  rescan  h  Jirogra nt .  \l  1 IHI l!’-82-8.  Vi  righi-l’aliei-on  \FB.  Oil:  Logistic-  and 
leehiueal  1  raining  Hivision.  \ i r  I  oree  Human  Resource-  Labor  an  jrv .  \prtl  l‘IJ',2 

V  olume  III 

Baum.  HR..  Modriek.  J.  V.  &  Hollingsworth.  S.R.  I  nun  trninin g  fur  t  iiiiiinuntl  tint!  mnlrul  systems: 
Hurt  mini  initial  it  ms  far  n/i/ilietilitm  of  rnrrimi  technology.  AFHItl  -TI>-!(;?-,l.  \\  righi-l’atler«oti  AKB. 
Oil:  Logistics  and  Technical  Training  Division.  Air  force  Human  Resources  Lahoratori.  April  H182. 

Volume  IA 

Hollingsworth.  S.R..  Modriek.  J.A..  &  Baum.  D.R.  Team  training  fur  command  and  control  systems: 
Recommendations  for  simulation  facility.  AFHRL-TP-82-10.  Wright-Paltersmi  AKB.  OH:  Logistics 
and  Technical  Training  Division.  Air  Force  Human  Resources  l.aboratorv.  April  1<)82. 

Volume  V 

Baum.  D.R..  Modriek.  J.A..  &  Hollingsworth.  S.R.  Team  training  f"r  command  and  control  systems: 
Executive  summary.  AF1IRI. -TP-82-1  1.  Wrighl-Patterson  \FB.  OH:  l.ogistics  and  Technical  Training 
Division.  Air  Force  Human  Resources  Laboratory.  April  I '182. 

This  paper  is  the  second  of  five  volumes  prepared  by  Honeywell  to  document  the  result-  of  a  research 
program  to  evaluate  the  current  status  of  team  training  (T2)  lor  operators  of  complex  Air  Force  command  anil 
control  (AFC2)  systems,  and  to  make  recommendations  for  enhancing  the  A  FLAT 2  process.  The  research  was 
performed  for  the  Air  Force  Human  Resources  Laboratorv  under  contract  FXitil  .>-7'M  ,-IMl2.’>.  This  research 
effort  supports  a  major  new  Air  Force  Human  Resources  Laboratory  (AFIIRI.)  research  and  development 
program  whose  primary  objective  is  to  improve  T2  technologies  in  areas  particular^  relevant  to  \ i r  Force 
combat  readiness.  Tile  program  objective  requires  the  establishment  of  a  baseline  data  base  on  how  T2  is 
currently  conducted  in  the  Air  Force:  how  it  is  developed,  implemented,  and  evaluated.  Because  Vir  Force 
teams  vary  greatly  in  size,  structure,  and  functions,  it  would  he  impractical  lo  collect  data  on  the  training 
provided  to  all  of  them.  Rather,  the  scope  of  ibis  research  effort  bad  to  be  directed  at  an  area  with  potential 
high  payoff  for  increased  combat  readiness  and  effectiveness.  The  area  of  command  and  control  was  chosen 
as  a  point  of  departure  for  the  research  because  C2  teams  tend  to  be  well-defined  structurally,  are  of 
manageable  sice,  and  perform  functions  highly  representative  of  Air  Force  mission  needs.  Furthermore,  as 
the  research  effort  unfolded;  limited  time  and  resources  made  if  necessary  to  focus  on  tactical  and  air  defense 
C*  systems  to  the  exclusion  of  strategic  C*  systems.  Thus,  the  C*  systems  surveyed  arc.  or  in  the  ease  of 
planned  systems  will  become.  Tactical  Air  Command  (TAG)  resources. 

The  goal  of  this  effort  was  to  develop  a  picture,  through  interview  and  observation,  id  how  AIC2T2  is 
currently  developed,  implemented,  and  evaluated,  and  what  C2  training  needs  will  arise  in  the  future.  Based 
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CHAPTER  I 


INTRODUCTION  AND  OVERVIEW 

2 

The  status  of  team  training  (T  )  programs  in  tactical  air  and  air  defense 

2 

command  and  control  (C  )  is  characterized  in  Volume  I  of  this  report. 

2 

The  methods  used  to  define,  develop,  implement,  and  evaluate  these  T 

programs  were  described,  analyzed,  and  evaluated.  The  description 

2 

was  based  on  survey  data  collected  through  interviews  with  C  training 
personnel  and  observations  of  training  in  Tactical  Air  Command  (TAC) 
and  Air  Training  Command  (ATC).  The  systems  surveyed  were  Airborne 
Warning  and  Control  System  (AWACS),  the  Semi-Automated  Ground 
Environment  (SAGE),  and  the  Tactical  Air  Control  System  (TACS).  The 
organizations  surveyed  in  TACS  were  the  Tactical  Air  Control  Center 
(TACC)  and  Control  and  Reporting  Center  (CRC).  Basic  and  Automatic 
Positionally  Qualified  (APQ)  training  programs  were  also  surveyed. 

The  analysis  and  evaluation  of  the  survey  data  permitted  the  identification 

2  2 

of  strengths  and  weaknesses  in  tactical  air  and  air  defense  C  T  .  The 
weaknesses  were  consolidated  into  issues  and  problem  areas  which  were 
grouped  into  the  categories  of  definition  and  development,  implementation 
and  evaluation,  and  modification  of  training  programs  and  training  devices 
or  personnel  policy  and  resource  constraints. 

These  issues  and  problem  areas  were  sorted  into  two  categories:  those 
amenable  to  solution  in  the  short  term  using  existing  technology  and 
those  requiring  more  research  and  solvable  in  the  long  term.  Some  of 


1 


the  issues  and  problems  can  be  addressed  by  both  approaches.  The  topics 
requiring  research  are  analyzed  in  this  volume  and  translated  into  recom¬ 
mended  objectives  and  projects.  The  analysis  of  the  topics  within  current 
technology  is  reported  in  Volume  HI. 

Sixteen  objectives  for  long-term  research  were  derived  from  the  issues 

2  2 

and  problem  areas  in  C  T  .  Both  the  issues /problems  and  objectives  are 
listed  in  Table  1.  Derivation  and  selection  of  the  research  objectives  were 
done  by  an  analysis  of  the  technical  literature  in  team  training  and 
information  gathered  during  our  survey  and  observations.  The  purpose 
of  this  volume  is  to  summarize  the  process  and  outcome  of  that  analysis. 

OVERVIEW  OF  THE  REPORT 

The  starting  point  is  a  brief  summary  of  the  status  of  team  training  as 

2 

found  in  our  survey  of  C  systems  or  organizations  in  tactical  air  control 

and  air  defense.  The  summary  is  followed  by  a  statement  of  problems 
2  2 

and  needs  in  C  T  . 

2 

The  operational  setting  of  the  C  organizations  visited  will  be  described 

for  the  purpose  of  identifying  an  operational  and  organizational  focus  for 

the  proposed  research.  The  focus  should  be  realistic  and  representative 

of  real-world  problems;  a  strategy  for  ensuring  that  these  conditions  are 

met  is  to  work  back  from  the  applied  setting  to  the  research  plan  and 

facility.  The  proposed  research  would  be  carried  out  in  a  simulation 

2 

facility  representing  operational  settings  for  control  of  tactical  air  C  , 
depending  on  the  specific  objectives  of  a  study.  The  operational  focus  will 
serve  as  a  basis  for  identifying  the  required  functional  capabilities  of  the 
simulation. 


RECOMMENDATIONS  FOR  LONG-TERM  RESEARCH 


The  state  of  knowledge  in  team  training  is  summarized  with  the  objectives 
of  identifying  major  themes  and  issues  and  evaluating  the  state  of  knowledge 
and  methodology  as  a  foundation  for  the  proposed  research.  There  have 
been  several  surveys  of  the  research  literature  on  teams,  team  training, 
and  small  groups  during  the  past  few  years.  It  has  also  been  assessed 
for  its  usefulness  in  addressing  practical  problems  of  team  performance 
and  training  and  found  to  fall  substantially  short  of  what  is  needed.  There 
is  little  need  to  survey  it  again  but  there  is  a  need  to  begin  some  critical 
interpretation  and  integration,  with  an  objective  of  developing  technology 
for  the  design,  management,  and  training  of  teams. 

Finally,  recommended  long-term  research  objectives  and  research  projects 
are  discussed.  The  specific  research  topics  are  rated  against  criteria  of 
feasibility,  utility,  probability  of  success,  and  the  availability  of  the 
facilities  and  skills  required.  Each  project  is  assigned  a  priority  based 
on  these  ratings  and  the  top  third  of  the  projects  is  recommended  for 
implementation. 
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CHAPTER  H 


BACKGROUND  TOPICS  RELATED  TO  RESEARCH  IN  AFC2T2 


The  purpose  of  this  chapter  is  to  summarize  topic  areas  which  provide  a 
frame  of  reference  for  developing  and  describing  specific  research: 

2 

•  Status  of  team  training  in  C  as  found  in  our  survey  and 
observations  during  site  visits 

2  2 

•  Summary  statement  of  problems  and  needs  in  C  T 

2 

•  Discussion  of  the  operational  settings  of  C  organization 
that  control  tactical  air  operations  for  the  purpose  of 
extracting  a  focus  on  a  representative  context  for  the 
research 

•  Discussion  of  the  status  of  knowledge  in  teams  performance 
and  training 

2  2 

The  domain  of  potential  C  T  research  topics  and  areas  is  large.  There 
is  substantial  literature  on  the  characteristics  and  operation  of  teams 
and  groups  as  indicated  by  recent  reviews  (References  1  through  5). 

The  recent  report  on  training  by  the  Defense  Science  Board  (Reference  6) 
has  given  an  impetus  and  higher  priority  to  research  on  team  training. 
Team  training  was  identified  as  a  major  weakness  and  a  high-priority 
research  program  was  recommended. 
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However,  the  amount  and  content  of  the  research  has  not  been  sufficient 
to  provide  a  systematic  body  of  knowledge  on  which  to  base  a  technology 
for  team  performance  and  training.  The  research  has  often  not  been 
focused  on  the  solution  of  problems  in  man-machine  operating  systems; 
however,  when  it  has  been  oriented  toward  these  problems,  the  laboratory 
tasks  and  constraints  have  been  seriously  unrepresentative  of  the  opera¬ 
tional  problems  and  conditions. 

STATUS  OF  TEAM  TRAINING  IN  C2 
2  2 

Our  survey  of  C  T  in  tactical  air  command  revealed  that  there  are  no 

formal  training  programs  for  team  training  in  command  and  control.  The 

2 

lack  of  formal  training  reflects  an  absence  of  T  requirements  and 
objectives  during  development  of  systems  and  exercise  programs  for 
maintenance  of  system  readiness.  There  is  no  requirement  to  describe 
and  report  job  tasks  for  teams  nor  to  do  a  task  analysis  to  provide 
requirements  for  human  engineering  design  or  training.  Systematic 
definition  and  development  of  training  for  teams  and  team  skills  is 
virtually  non-existent.  There  is  no  formal  training  for  supervisory  and 
battle  staff  personnel,  who  are  critical  to  the  effective  operation  of  teams. 

Simulation  is  used  for  systems  like  AWACS  and  CBC/CRP  but  the  number 

and  capability  of  these  devices  is  less  than  adequate.  The  simulation 
2  2 

facilities  in  AFC  T  are  significantly  behind  the  state  of  the  art;  the 

2  2 

lack  of  adequate  data  on  C  T  jobs,  tasks,  and  training  requirements 
significantly  impairs  the  definition  of  needs  and  capabilities  for  simulation. 
Facilities  which  can  provide  exercise  of  units  as  integrated  teams  and 
joint  exercises  of  two  or  more  units  or  commands,  such  as  Blue  Flag, 
have  serious  limitations. 
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This  summary  is  brief  and  global;  an  extended  and  detailed  treatment  is 

contained  in  Volume  I.  It  is  sufficient  to  substantiate  the  point  that  the 
2  2 

level  of  AFC  T  is  less  than  what  is  needed  and  what  is  possible  within 

the  state  of  the  art.  Furthermore,  there  are  many  opportunities  for 

technical  development  and  research  to  make  contributions  which  can 

increase  training  effectiveness,  improve  the  design  of  man-machine 

2 

interfaces  and  personnel  subsystems  in  C  systems,  reduce  costs,  and 
increase  system  effectiveness.  There  are  major  needs  in  the  development, 
implementation,  and  evaluation  of: 

•  Team  training 

•  Performance  measurement 

•  Simulation 

•  Tools  for  design  and  analysis 

•  Instructional  strategy 

SUMMARY  OF  PROBLEMS  AND  NEEDS  IN  AFC2T2 
2  2 

The  state  of  AFC  T  is  summarized  in  a  set  of  issues  and  problem  areas 
presented  in  Table  1.  They  are  discussed  in  Volume  I.  They  are  grouped 
into  four  categories : 

•  Definition  and  development 

•  Implementation 

•  Program  evaluation  and  modified  action 

•  Personnel  policy  and  resource  constraints 
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Many  of  the  issues  and  problems  have  been  mentioned  or  alluded  to  in  the 
preceding  discussion.  The  lack  of  a  definitive  framework  for  team 
phenomena  is  listed  under  definition  and  development  but  it  is  applicable 
to  all  areas.  The  lack  of  an  agreed-upon  and  standardized  terminology 
and  concepts  is  an  impairment  for  typical  system  personnel  in  recognizing, 
reporting,  and  analyzing  team  behaviors  and  in  planning  and  implementing 
supporting  programs.  The  resulting  lack  of  technology  weakens  all  related 
areas  of  human  factors:  design  of  man-machine  interfaces,  design  of 
team  structures,  identification  of  personnel  requirements  and  qualifications, 
performance  measurement,  simulation  and  training.  Thus,  the  design  of 
systems  for  effective  use  and  support  of  teams  of  operators  is  seldom 
adequate  except  by  trial-and-error  evolution. 

Measurement  and  evaluation  issues  are  a  recurring  topic  with  an  emphasis 
on  the  need  for  criteria  and  standards.  The  need  for  procedures, 
techniques,  methods,  and  supporting  data  is  a  frequent  theme.  The  need 
is  typically  not  one  for  developing  methodology;  existing  human  factors 
methods  are  probably  adequate  if  adapted  to  the  application  to  team  design 
and  if  team  phenomena  are  appropriately  structured. 

Several  of  the  issues  and  problems  can  be  alleviated  with  relative  ease  if 
resources  are  made  available,  even  though  definitive  solutions  might 
require  additional  research.  For  example,  courses  and  instructions  can 
be  provided  to  meet  the  specific  training  needs  of  instructors  in  operations 
training,  supervisors  and  battle  staff,  AWACS  weapons  controllers,  and 
personnel  controlling  simulated  aircraft.  The  mismatch  of  the  syllabus 
for  the  Air  Weapons  Controller  Fundamentals  Course  and  the  entry 
requirements  into  this  course  and  career  field  can  be  corrected. 
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The  incomplete  use  of  existing  evaluative  data  from  exercises  and  after¬ 
action  reports  (AARs)  can  be  improved  by  developing  or  adapting  methods 
and  procedures  for  collecting,  analyzing,  reporting,  and  interpreting 
performance  data  for  individuals,  teams,  units,  and  systems.  This  effort 
might  be  a  more  intermediate  than  a  short-term  effort,  however,  since 
it  is  dependent  on  developments  in  establishing  criteria  and  measures. 

It  might  also  entail  significant  costs  in  automated  data  processing  to 
handle  the  potential  large  body  of  data  and  provide  computer  aiding  for 
the  evaluators  and  analysts. 

Similarly,  the  use  of  simulation  could  be  greatly  improved  by  applying  the 

state  of  the  art  in  simulation  technology,  modeling,  and  war  gaming. 

Developments  in  the  areas  have  made  it  feasible  to  achieve  much  greater 

capability  and  useful  fidelity  in  simulation  and  to  achieve  it  at  greatly 

reduced  costs  per  function.  These  trends  will  continue  with  development 

of  technology  in  integrated  circuits  and  it  is  important  to  project  the 

research  to  meet  the  needs  of  future  systems.  However,  it  is  important 

2  2 

to  structure  adequately  the  C  T  simulation  requirements.  Structuring 

2 

will  require  an  understanding  of  team,  C  ,  the  threat  tactical  doctrine, 
the  functional  characteristics  of  the  systems,  and  the  person-system 
transactions.  Failure  to  structure  the  application  adequately  falls  under 
the  old  computer  axiom  of  "garbage  in,  garbage  out.  " 

The  set  of  issues  and  problems  under  Definition  and  Development  (Table  1) 
is  fundamental  in  many  ways  to  the  other  areas.  Research  on  them  will 
address  needs  for  knowledge  and  techniques  which  must  be  resolved  before 
progress  can  be  made  in  the  other  areas.  In  addition,  they  will  have 
wide  applicability. 
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The  problem  area  of  instructional  methods  for  team  training  may  seem  to 
be  superfluous  since  there  is  a  substantial  technology  for  instruction. 
However,  this  technology  has  been  developed  and  evaluated  for  individual 
training.  The  methods  may  be  applicable  directly  or  with  modification 
to  team  training  but  there  is  little  data  to  that  effect.  The  first  step  in 
developing  a  technology  for  team  training  is  to  review  the  methods  for 
individual  training  for  applicability.  Then  methods  and  strategy  specifi¬ 
cally  for  team  training  should  be  investigated. 

Personnel  Policy  and  Resource  Constraints  (Table  1)  is  a  bothersome 

area  because  it  contains  issues  that  have  a  major  impact,  often  negative, 

but  there  may  be  little  that  can  be  done  about  them  through  research  or 

technological  applications  in  human  factors  and  training.  Studies  can  be 

done  to  generate  information  on  the  effect  or  causes  of  a  shortage  of  live 

flying  events,  loss  of  experienced  personnel,  and  poor  understanding  of 
2 

C  training  by  program  managers.  But  changing  those  conditions  and 
constraints  is  difficult  because  the  policies,  allocations  of  resources, 
and  constraints  are  the  responsibility  of  persons  in  other  areas  who  must 
also  weigh  other  priorities.  One  can  develop  and  recommend  alternative 
man-machine  configurations  and  manpower  practices  to  alleviate  the 
effects  of  the  constraints.  One  can  also  use  research  results  to  recommend 
and  lobby  for  changes.  But  the  decision  falls  under  the  charter  of  someone 
in  another  chain  of  command. 

That  pessimistic  observation  leads  to  a  concluding  comment  and  caveat 
about  recommending  a  research  program.  The  comment  is  that  a  research 
program  which  is  successful  in  the  long  run  must  consist  of  solving  problems 
of  applications,  managing  political  relationships  to  generate  a  supporting 
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consensus,  and  lobbying/ selling  to  have  an  effect  on  how  things  are  done 
and  the  cost-effectiveness  of  doing  them.  Human  factors  has  been  outside 
the  engineering  design  process  but  must  ultimately  become  integrated  into 
it.  "Good"  research  in  a  parochial  sense  is  not  sufficient  although  it  may 
be  necessary  for  success. 

The  caveat  is  that  the  subsequent  discussion  of  research  issues  and 
recommendations  will  deal  with  the  research  options  primarily  in  terms 
of  parochial  criteria  as  the  state -of- knowledge,  theory,  and  experimental 
feasibility.  The  discussion  has  an  applications  orientation  as  a  source 
of  the  problems  to  address.  However,  the  issues  of  consensus  and  having 
an  impact  on  the  real  world  of  systems  and  use  of  manpower  involve 
complex  processes  depending  on  local  conditions.  They  are  outside  the 
scope  of  this  effort. 

OPERATIONAL  SETTING  FOR  C2  TEAMS  IN  TACTICAL  AIR  COMMAND 

2 

The  objective  for  this  topic  is  to  define  the  domain  of  C  team  activities 
and  settings  for  the  purpose  of  focusing  on  or  selecting  a  set  of  experi¬ 
mental  conditions  for  this  research.  Hie  conditions  should  be  represent- 
2 

ative  of  AFC  activities  and  provide  the  capability  to  generate  behaviors, 

tasks,  variables,  and  constraints  such  that  results  of  studies  can  be 

interpreted  in  terms  of  implications  for  operational  performance.  The 

level  of  examination  is  largely  an  organizational  and  functional  description 
2 

of  C  systems.  The  organizations  described  are  from  Tactical  Air 
Command  and  therefore  are  in  the  operational  area  of  air  defense  and 
tactical  air  operations.  A  comparable  description  of  other  operational 
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areas  should  be  undertaken  before  one  attempts  to  generalize  the  tactical 
air  context  to  other  situations. 

The  organizations  in  TAC  ground  operations  discussed  are: 

•  Control  and  Reporting  Center  (CRC/CRP) 

•  Semi-Automated  Ground  Environment  (SAGE) 

•  Airborne  Warning  and  Control  System  (AWACS) 

•  Tactical  Air  Control  Center  (TACC) 

These  are  the  systems  with  which  we  are  familiar  as  a  result  of  our 

2 

survey;  they  were  chosen  to  be  representative  of  a  major  area  of  AFC  . 

The  aircraft  of  the  squadrons  which  they  support  are  also  integral  parts 
of  their  operational  activities.  The  personnel  and  activities  of  the 
organizations  provide  a  variety  of  teams  and  applications. 

2 

There  are  other  C  -related  organizations  in  TAC.  The  Wing  Operations 

Center  (WOC),  for  example,  manages  tactical  units  in  the  executions  of 

missions.  The  personnel  in  the  WOC  also  do  detailed  mission  planning 

and  defense  analysis.  The  Air  Support  Operations  Center  (ASOC)  processes 

requests  for  air  support.  It  coordinates  the  provision  of  close-air  support 

with  Army  maneuver  units  through  the  Tactical  Air  Control  Parties  (TACPs). 

Scramble  orders  are  prepared  in  response  to  the  requests  and  transmitted 

2 

to  the  WOC  for  execution.  These  activities  are  clearly  C  ground 
operations  in  support  of  air  operations. 


12 


However,  we  believe  that  the  functions,  jobs,  and  tasks  in  the  selected 

2 

systems  are  representative  of  those  in  the  other  organizations.  All  C 
systems  perform  some  subset  of  activities  such  as  data  acquisition  and 
analysis,  planning,  resource  allocation,  detecting  and  evaluating  threat, 
executing  or  implementing  an  operational  plan,  and  so  on.  Knowledge 
of  the  specific  situation  and  local  boundary  conditions  is  necessary  for 
application  but  it  is  assumed  that  component  tasks  and  behavior  in  these 
activities  are  the  same  or  equivalent. 

There  are  also  large-scale  joint  exercises  involving  several  organizations 
and  very  large  scenarios.  They  are  also  assumed  to  consist  of  an 
aggregation  of  functions  and  activities  of  component  elements.  They  are 
too  complex  to  be  the  starting  point  for  a  research  program.  They  may 
be  an  objective  for  later  phases.  However,  we  should  first  get  a  better 
comprehension  of  the  problems  and  issues  of  the  components. 

Studies  and  development  of  joint  exercises  could  be  undertaken  on  an 
ad  hoc  basis  because  a  systematic,  man-machine  approach  can  make 
several  contributions  within  the  current  state  of  the  art.  They  are  not 
discussed  here  because  they  are  systems  engineering  rather  than  research. 

2 

A  research  program  in  team  training  for  C  teams  must  focus  on  some 
part  of  all  the  types  of  activities  and  tasks  in  these  systems.  Only  such 
a  focus  will  ensure  that  realistic  conditions  and  constraints  are  represen¬ 
ted  in  the  research  plans.  Thus,  the  research  plan  must  be  built  around 
problems  and  situational  contexts  in  these  organizations. 
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Each  organization  will  be  described  briefly  to  give  an  overview  of  the 
kinds  of  activities  and  factors  relevant  to  selecting  an  appropriate  vehicle 
for  research. 


Control  and  Reporting  Center /Control  and  Reporting  Post 

The  purpose  of  these  organizations  is  to  control  and  manage  air  traffic 
in  an  assigned  region  of  air  space.  These  areas  of  responsibility  consist 
primarily  of  directing  aircraft  flying  on  air  defense  against  hostile 
aircraft,  close-air  support  for  ground  operations,  interdictions,  defense 
suppression,  and  reconnaissance.  Air  defense  consists  of  protecting 
friendly  aircraft  on  the  other  missions  as  well  as  protecting  ground 
targets;  it  may  be  against  fighter  and  bomber  aircraft.  Air  refueling 
is  an  important  special  operation  in  these  activities. 

The  direction  of  intercepts  by  a  Weapons  Director  (WD)  and  technician 
is  the  principal  function.  The  operation  of  a  CRC  and  CRP  are  basic 
examples  of  the  implementation  of  this  function.  The  pilot  of  an 
intercepter,  WD,  and  technician  form  a  team  for  the  purposes  of 
executing  an  intercept  course  from  which  an  attack  can  be  flown  against 
the  hostile.  The  WD  follows  the  hostile  and  interceptor  aircraft  on  a 
radar  scope.  He  gives  the  pilot  continuously  updated  range  and  bearing 
on  the  hostile  and  heading  for  interception.  The  intercepters  may  fly 
from  either  an  airbase  or  the  position  of  a  Combat  Air  Patrol  aircraft, 
assigned  to  a  region  of  the  airspace  near  an  anticipated  avenue  of 
approach  of  a  threat. 


The  weapons  direction  is  supported  by  functions  of  targeting  and 
surveillance.  Targeting  consists  of  differentiating  among  friendly  and 
hostile  tracks,  assessing  the  priority  of  threat,  and  assigning  interceptors 
to  threats  in  accordance  with  the  rules  of  engagement.  For  example,  the 
rules  of  engagement  in  one  scenario  were:  engage  hostile  aircraft  at 
maximum  range,  engaging  high  threat  targets  first,  then  aircraft  emitting 
electronic  countermeasures  (ECM),  and,  finally,  any  other  unengaged 
hostile  targets.  High  threat  targets  were  unengaged  targets  near  critical 
areas  (CRC,  CRP,  TACC,  air  bases,  and  cities).  Surveillance  consists 

^  of  detecting  and  reporting  unknown  tracks  on  a  radar  scope.  A  surveillance 

operator  and  technician  are  usually  assigned  responsibility  for  surveillance 
in  a  region  of  the  airspace. 

A  battle  staff  provides  supervision,  coordination,  and  support  of  these 
activities.  Their  responsibility  is  management  of  the  air  battle  in  their 
assigned  portion  of  the  airspace.  Ihey  must  allocate  and  deploy  resources 
in  accordance  with  the  operational  order  and  developing  tactical  situation. 
They  provide  the  leadership  for  teams. 

,  The  "picture"  of  the  air  battle  is  depicted  on  a  large  vertical  plastic 

display  surface  on  which  alphanumeric  and  graphic  data  are  manually 
recorded  from  the  back  surface  of  the  display.  The  tracks  of  bogey 
flights  and  interceptors  are  plotted  on  a  geographic  map.  Information 
about  scheduled  flights  and  flight  status  is  maintained  in  tabular  form. 

Plots  are  maintained  by  two  sets  of  people:  tellers  who  sit  in  front  of 
the  display  at  the  section  for  which  they  are  responsible  and  plotters 
behind  the  display.  Data  to  be  plotted  are  given  to  the  tellers  in  written 
form  or  voice;  they  relay  the  information  in  voice  by  telephone  to  the 
plotters . 
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This  large  display  contains  the  information  used  by  the  battle  staff  and 
their  support  personnel.  The  surveillance  operators  and  WDs  have 
automated  situational  moving-target-indicator  displays  on  their  radar 
scopes.  Typically,  the  user  has  selected  only  the  limited  geographic 
area  for  which  he  is  responsible  and  he  does  not  look  at  the  total  battle. 
These  displays  are  not  provided  for  the  battle  staff  and  not  used  by  them. 

Semi-Automated  Ground  Environment 

The  operations  of  SAGE  are  very  much  like  those  of  the  CRC  and  CRP. 

The  roles  of  the  systems  in  air  defense  differ.  SAGE  is  responsible  for 
continental  air  defense  in  conjunction  with  NORAD;  the  CRC  and  CRP  are 
responsible  for  air  defense  in  support  of  tactical  operations  in  the  airspace 
of  a  combat  theater. 

SAGE  has  larger  numbers  of  personnel  and  computer  support  for  most 
operator  and  command  positions;  however,  it  is  also  somewhat 
obsolescent,  having  been  designed  initially  to  the  state  of  the  art  in 
computers,  sensors,  and  weapons  available  in  the  period  of  the  1940's 
and  1950's.  It  is  being  phased  out  to  be  replaced  by  the  Joint  Surveillance 
System  ( JSS).  Further  information  on  SAGE  can  be  found  in  References 
7  and  8. 

Airborne  Warning  and  Control  System 

AWACS  is  similar  in  function  to  the  CRC  and  SAGE.  It  is  different  in 
two  ways:  it  is  airborne  rather  than  ground-based,  and  its  air  traffic 
control  staff  is  smaller.  The  radar  and  computer  equipment  are  also 
built  with  more  recent  electronic  technology. 
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Tactical  Air  Control  Center 


The  TACC  is  an  integration  and  planning  center  using  entirely  manual 
operation  in  its  present  configuration.  The  manual  vertical  plotting 
board  described  earlier  is  the  situational  display.  Some  sections,  such 
as  intelligence,  use  an  acetate  overlay  and  grease  pencil  for  analysis 
and  planning.  Information  and  messages  in  and  out  of  TACC  are 
transmitted  over  voice  channels. 

The  activities  in  TACC  are  heavily  dependent  on  knowledge  of  terminology, 
operating,  and  reporting  procedures  in  a  multi-national  force  structure 
and  limitations  of  equipment  and  logistics.  This  characterization  can  be 
illustrated  by  the  following  list  of  training  objectives  for  Blue  Flag  80. 1: 

2 

•  Be  familiar  with  fundamentals  of  C 

2 

•  Know  the  NATO  2  ATAF  C  structure 

•  Understand  theater  terminology 

•  Know  NATO/USAFE  reporting  procedures  and  report  formats 

•  Understand  COMSEC/OPSEC  procedures 

•  Understand  the  Warsaw  Pact  threat  and  friendly  force 
counterthreat 

•  Know  the  NATO  stages  and  states  of  alert 

•  Be  familiar  with  theater  documents 

•  Be  familiar  with  vulnerabilities  and  limitations  of  allied 
communications  systems 

•  Be  familiar  with  logistics  operations  and  limitations 
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Some  automation  of  the  TACC  is  in  progress  and  development.  An  initial 
attempt  to  develop  an  automated  TACC,  the  485L  program,  has  been 
abandoned.  The  current  attempt  consists  of  developing  minicomputer 
support  systems  to  aid  specific  functions  within  the  center.  The 
Computer-Aided  Force  Management  System  (CAFMS),  for  example,  is 
an  aid  for  resource  allocation  developed  by  the  Honeywell  Electronic 
Systems  Division  (ESD)  and  being  implemented  and  evaluated  by  the 
4442nd  Test  Squadron  at  Eglin  AFB  in  Blue  Flag  exercises.  Other 
support  systems  under  development  at  ESD  are: 

•  Force  Management  Enhancements  (FME) — Various 
decision-aiding  techniques  for  planning  and  commitment, 
primarily  resource-to-target  assignment.  It  was  formerly 
called  Tactical  Operations  Planner  or  TOP. 

•  Knowledge -Based  System  (KNOBS) --An  artificial  intelligence 
processing  technique  known  as  "production  rules"  for 
weapon-to-target  assignment.  It  has  the  capability  to 
change  rules  for  target  assignment  in  the  field  without 
reprogramming.  It  also  permits  the  user  to  trace  the 

logic  behind  a  trial  assignment  to  determine  whether  he 
concurs,  and  if  not,  to  modify  the  assignment. 

Thus,  the  nature  of  the  TACC  will  be  undergoing  major  change.  Research 
on  team  activities  in  the  current  manual  TACC  will  be  inappropriate 
because  many  of  them  will  soon  be  modified  significantly. 

The  recommended  operational  settings  on  which  to  base  a  research 

program  in  team  training  are  the  CRC,  CRP,  and  AWACS.  They  encompass 

2 

a  representative  range  of  types  of  teams  and  C“  activities  within  common 
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tactical  missions.  The  situations  are  also  stable  enough  to  serve  as  an 
external  reference  for  designing  and  evaluating  research  studies. 

The  TACC  is  not  an  appropriate  setting  in  its  present  form  because  it 
will  be  undergoing  major  changes.  Neither  the  scope  nor  the  impact  of 
these  changes  is  yet  known.  Research  studies  could  be  done  on  the 
next-generation  TACC  by  extrapolating  current  knowledge  and  plans  and 
anticipating  future  problems.  The  worth  of  such  an  approach  is  dependent 
on  the  adequacy  of  the  data  base  which  one  extrapolates;  its  adequancy 
is  presently  unknown. 

Research  on  computer  support  and  aiding  for  TACC-type  activities  would 
be  more  appropriate.  It  could  provide  information  on  what  activities 
need  support  and  how  to  aid  them.  The  research  would  also  contribute 
to  formulating  concepts  of  the  next-generation  TACC. 

SAGE  is  not  an  appropriate  vehicle  since  it  is  obsolescent  and  being 
phased  out  of  the  inventory.  The  development  status  of  the  JSS,  the 
replacement  for  SAGE,  would  have  to  be  assessed  for  suitability. 

The  CRC/CRP  functions  and  personnel  structure  should  be  broken  down 

to  identify  the  teams  and  structure  of  teams  within  it.  Of  course,  the 

2 

description  of  a  team  must  include  C  activities.  The  selection  of  teams 
and  conditions  to  simulate  and  issues  to  investigate  can  be  made.  The 
selection  would  be  influenced  by  local  circumstances  of  a  supporting 
consensus  and  availability  of  resources  as  well  as  research  criteria. 
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STATE  OF  KNOWLEDGE  IN  TEAM  PERFORMANCE  AND  TRAINING 


The  purpose  of  this  topic  is  to  summarize  or  characterize  the  state  of 

knowledge  in  the  area  of  team  research.  The  objective  is  to  provide  a 

preliminary  framework  for  thinking  and  discussion  in  the  context  of 
2 

military  C  activities  rather  than  a  comprehensive  analysis  of  the 
research  literature.  The  ideas  expressed  are  an  outgrowth  of  reflections 
on  the  research  literature  and  the  content  of  the  reviews. 

The  needs  for  knowledge  about  performance  and  training  of  teams  are 
many  and  broad;  further,  the  state  of  the  existing  knowledge  is  so 
unstructured  that  better  focusing  and  delineation  of  problems  is  a 
necessary  first  step.  This  literature  has  been  reviewed  several  times 
in  recent  years  (References  1-5).  There  is  a  consensus  among  investi¬ 
gators  on  global  areas  or  topics.  The  general  findings  have  been  that 
the  amount  and  content  of  the  research  has  been  insufficient  to  provide 
a  systematic  body  of  knowledge  on  which  to  base  a  technology  for  team 
performance  and  training,  especially  in  military  applications.  The 
research  has  often  not  been  focused  on  the  solution  of  problems  in 
man-machine  operating  systems.  When  it  has  been  oriented  toward 
these  problems,  however,  the  laboratory  tasks  and  constraints  have 
been  seriously  unrepresentative  of  the  operational  problems  and 
conditions.  Goldin  and  Thorndyke  (Reference  4)  summarized  the 
consensus  among  participants  in  a  workshop  on  team  performance 
consisting  of  eight  sessions.  The  issues  that  emerged  were  subsumed 
under  the  topics  of  defining  team  characteristics  in  relation  to  team 
effectiveness,  organizational  constraints  and  determinants  of  performance 
requirements,  team  structure,  communication,  and  training  techniques 
for  teams. 
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Representative  conclusions  are  presented  by  Collins  (Reference  2) 
following  a  review  of  research  on  small  groups  for  potential  contribution 
to  team  training.  He  identified  needs  in  understanding  of  group  interac¬ 
tions,  technical  and  social  dimensions  of  team  tasks,  variables  and 
relationships  in  the  input-process -performance  of  team  tasks,  the 
compatibility  between  individual  and  group  goals  and  how  they  are  set, 
the  roles  of  hedonistic  individual  orientation  and  altruistic  commitment 
to  a  group,  and  processes  for  integrating  individual  performance  into 
team  accomplishment.  This  list  is  a  very  broad  and  comprehensive 
coverage  of  this  area  of  research. 

Definition  of  a  Team 

There  are  several  definitions  that  differ  from  each  other  in  the  attributes 
and  properties  chosen.  However,  a  statement  of  the  minimum  conditions 
necessary  for  a  team  to  exist  is  a  first  step.  There  is  a  consensus  in 
the  literature  that  all  teams  must  have  the  following  properties  as  a 
minimum: 

•  An  objective,  goal,  or  mission  toward  which  the  team  is 
working.  This  objective  implies  a  product  or  output  by 
which  the  adequacy  of  team  performance  can  be  evaluated. 

•  Roles  assigned  to  individual.  Roles  are  the  functions  or 
activities  that  are  the  responsibility  of  the  individual. 

They  represent  a  division  of  labor  among  team  members. 

Each  role  contributes  something  to  the  product  of  the 
team  or  control  of  the  team  processes.  Supervisory  and 
leadership  roles  are  predominantly  in  the  control  functions. 
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The  roles  vary  along  a  continuum  of  homogeneity-heterogeneity 
functions.  Increasing  heterogeneity  is  accompanied  by 
increasing  interdependence.  The  functions  of  teams  with 
heterogeneous  roles  are  partitioned  among  team  members 
into  tasks  that  are  very  different;  for  example,  piloting, 
intercept  direction,  and  target  detection.  Interdependence 
is  high  because  each  team  member  has  an  essential  part 
of  the  team's  function  which  cannot  be  adequately  compen¬ 
sated  for  if  missing  or  below  standard.  Homogeneity  and 
low  interdependence  exist  in  a  surveillance  function  where 
multiple  operators  perform  the  same  task  of  detecting 
targets  in  separate  geographical  areas. 

•  Structure.  The  positions  in  the  team  are  an  interconnected 
network.  The  team  can  be  viewed  as  a  distributed  processing 
network  of  stations,  each  contributing  an  increment  to  the  flow 
of  work.  The  structure  has  two  segments:  production  and 
control /management. 

•  Internal  communication  among  members  of  the  team.  The 
coordination  and  control  of  team  activities  requires 
communication . 

These  are  minimal  conditions  and  they  are  highly  interrelated.  Many 
different  kinds  of  teams  can  exist  within  these  four  conditions.  This 
framework  needs  to  be  supplemented  with  concepts  and  variables  that 
permit  differentiating  between  different  teams  and  team  activities. 
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One  capability  that  must  be  achieved  is  to  be  able  to  differentiate  among 
team  and  individual  behaviors  and  the  behavior  of  multiple  individuals 
acting  independently  or  in  parallel.  An  apparent  interrelatedness  of 
behaviors  among  individuals  can  be  produced  by  a  common  external 
driver  which  acts  to  synchronize  behaviors  that  are  in  fact  independent 
of  each  other.  The  intuitive  idea  of  a  team  includes  a  sense  that  the 
team  is  more  than  a  collection  of  individuals.  It  includes  an  interde¬ 
pendence  and  intentional  interaction  for  the  purpose  of  working  together 
toward  a  common,  shared  objective.  The  mechanism  of  the  interaction 
has  not  been  identified,  however. 

It  is  commonly  stated  in  the  technical  literature  that  teams  must  have  a 
shared  goal;  this  is  a  necessary  property  of  a  good  team.  However,  we 
believe  that  it  is  more  important  that  teams  have  a  shared  plan  or  schema. 
The  shared  plan  is  an  internalized  (mental)  scheme  for  organizing  and 
integrating  team  activities  in  performing  the  operational  tasks  required 
to  accomplish  their  mission  goal.  The  goal  is  usually  imposed  by  an 
external  agency  in  the  form  of  a  required  output  and  it  provides  little 
information  for  the  team  process  beyond  scoring  output  of  the  team's 
performance.  However,  the  shared  plan  can  be  used  by  individual 
members  of  the  team  to  adjust  their  behaviors  to  the  needs,  style, 
competence,  and  other  characteristics  of  team  members.  The  shared 
plan  connects  the  operational  tasks  and  team  behaviors. 

Team  Performance  Dimensions 

The  activities  of  a  team  can  be  separated  into  two  types:  operational 
tasks  and  team-oriented  behaviors.  Ibis  distinction  is  not  usually  made 
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explicitly  in  the  literature  although  team  behaviors  and  skills  are  assumed 
and  discussed. 

A  team  is  a  decernible,  functional  unit;  it  is  a  set  of  two  or  more  individuals 
who  must  behave  in  a  coordinated  manner  to  perform  some  work  measurable 
•as  a  countable  output.  The  most  visible  and  concrete  activities  of  a  team 
are  the  operational  tasks  that  achieve  the  team's  output.  The  operational 
tasks  are  related  to  the  mission  or  objective  of  the  team.  The  objective 
is  achievement  of  an  output  or  product.  It  is  measured  by  indicators  such 
as  percent  kills,  attrition,  survivability,  and  objectives  gained,  for 
example.  Performance  on  these  tasks  is  measured  primarily  through 
product  measures. 

However,  there  is  a  second  set  of  team-oriented  behaviors  that  includes 
the  ways  in  which  the  team  members  interact  and  relate  while  performing 
the  operational  tasks.  These  behaviors,  rather  than  the  operational  tasks, 
differentiate  a  team  from  a  collection  of  individuals  working  independently. 
They  must  be  differentiated  from  the  operational  tasks  and  measured 
separately  from  those  performance  measures.  The  team-oriented 
behaviors  are  the  behaviors  of  interactions,  coordination,  and  integration 
that  constitute  the  operating  plan  or  schema  that  the  team  members  use 
to  accomplish  their  operational  tasks.  The  team  schema  may  include 
several  modes  corresponding  to  different  operational  conditions. 

The  performance  of  both  the  operational  tasks  and  team  behaviors  must 
be  evaluated  in  terms  of  both  product  and  process  measures.  Product 
refers  to  a  quantitative  accomplishment  or  output  while  process  is  a  more 
qualitative  indicator  of  how  the  product  is  achieved.  For  example,  an 
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operational  product  measure  might  be  the  proportion  of  intercepts 
accomplished;  a  process  measure  might  be  how  the  weapons  controller 
established  priorities.  The  adjustment  and  equalization  of  workload 
among  team  members  might  be  a  product  measure  in  team  behaviors, 
and  the  speed  of  recognition  or  anticipation  of  changes  in  workload  might 
be  a  process  measure.  The  identification  of  product  and  process  events 
in  team  functioning  is  needed  to  clarify  the  behavioral  processes  in  the 
functioning  of  teams. 

The  contribution  of  team  behaviors  to  accomplishing  the  objective  of  the 
team  is  unknown.  It  is  generally  assumed  that  some  level  of  mastery  of 
team  behaviors  is  necessary  for  higher  levels  of  performance  on 
operational  tasks.  Extensive  technical  knowledge  alone  is  not  sufficient 
for  a  team  to  function  well;  it  is  also  doubtful  that  competence  in  team 
behaviors  alone  will  be  adequate  for  performance  of  operational  tasks. 
However,  there  is  no  empirical  evidence  to  support  such  an  assumption. 
The  interaction  between  these  two  behavioral  classes  needs  to  be 
determined. 

Current  practices  in  team  training  do  not  emphasize  these  team  compe¬ 
tencies;  they  are,  in  fact,  exercises  in  the  operational  tasks.  There 
seems  to  be  an  assumption,  at  least  by  default,  that  team  competence 
must  emerge  via  working  together  on  the  operational  tasks.  Some  good 
teams  will  emerge  but  such  an  unsystematic  approach  is  inefficient. 


25 


1 


A  complicating  factor  is  that  team  behaviors  cannot  be  observed  separately. 
One  cannot  observe  a  team  just  being  a  team;  it  must  be  doing  some 
operational  tasks.  Since  the  competence  of  a  team  is  a  product  of  the 
level  of  mastery  of  both  the  operational  tasks  and  team  behaviors,  the 
evaluation  of  deficiencies  in  performance  requires  that  the  two  components 
be  differentiated  in  the  flow  of  behavior  and  assessed  separately  in  order 
to  identify  the  source  of  poor  performance. 

Team  Identity 

This  discussion  treats  the  team  as  a  collection  of  individuals.  Our 
observations  and  analyses  have  led  repeatedly  to  the  conclusion  that  the 
interactions  between  team  members  depend  upon  individuals  who  have 
skilled  behaviors  which  they  use  in  reaction  to  cues  generated  by  team 
members  or  situational  events.  The  team  behaviors  are  individual 
behaviors  integrated  through  the  shared  plan.  In  contrast,  it  is  a 
common  assumption  that  a  team  exists  as  an  emergent  entity  and 
develops  an  identity  as  a  team.  Its  performance  is  assumed  to  improve 
as  this  identity  grows. 

If  such  an  identity  exists,  it  could  serve  as  a  useful  index  of  the  level  of 
team  development.  The  effect  of  the  team  identity  on  the  individual  would 
also  be  an  important  relationship  in  understanding,  designing,  and 
training  teams.  The  growth  of  a  team  identity  may  be  correlated  with 
level  of  understanding  of  the  shared  team  plan.  However,  team  identity 
as  a  causal  factor  on  team  performance,  rather  than  or  in  addition  to 
individual  factors,  should  be  viewed  with  skepticism. 
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The  individual  team  member  does  not  lose  his  identity  or  accountability 
in  a  team  setting.  The  skills  in  team  behaviors  are  individual  skills; 
at  least  there  is  no  data  that  invalidates  this  statement.  Coordination 
and  timing,  for  example,  are  behaviors  that  require  an  individual  to  use 
sensory  cues  to  perceive  conditions  and  adjust  his  behavior  to  them.  A 
skillful  team  member,  we  contend,  is  a  person  whose  experience  enables 
him  to  adjust  to  a  wide  variation  in  team  settings,  strategies,  and  work 
styles  of  team  members. 

The  emergence  of  a  team  identity  is  associated,  however,  with  a  number 
of  attributes  which  group  dynamicists  have  been  pursuing  for  years. 

They  include  attributes  like  cohesiveness,  loyalty,  and  belonging.  They 
are  manifested  in  commonplace  behaviors  of  teams  such  as  adopting 
common  articles  of  clothing,  insignia,  in-group  references,  jargon, 
and  codes  of  conduct.  Team  members  set  performance  standards  and 
exercise  an  internal  discipline  for  the  good  of  the  team.  Commitment 
to  the  group  sometimes  competes  with  personal  goals  and  values. 

These  factors  may  not  be  relevant  to  team  training,  design,  or  proficiency 
maintenance.  It  is  improbable  that  building  these  attributes  is  sufficient 
to  produce  adequate  team  behavior.  Rather,  these  attributes  develop 
after  the  basic  team  behaviors  and  operational  tasks  have  been  mastered 
to  a  level  where  the  team  exists  as  a  functional  unit.  Therefore,  they 
might  be  useful  as  indicators  for  assessment  of  team  development  or  to 
differentiate  between  well-  and  poorly-performing  teams. 
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Types  of  Teams 

A  final  general  issue  is  the  existence  of  different  types  of  teams.  The 
idea  that  there  would  be  only  one  kind  of  team  is  counter-intuitive. 

Teams  having  different  numbers  of  members  and  different  configurations 
of  communication  links  would  undoubtedly  differ  in  the  relative  frequency 
and  importance  of  the  team  competencies  and  behaviors. 

Teams  should  be  differentiable  in  terms  of  their  standing  on  independent 
dimensions.  There  seems  to  be  a  tendency  in  the  literature  to  treat  all 
teams  as  either  the  same  in  some  existential  sense  or  as  different  and 
unique.  However,  it  should  be  possible  to  partition  the  domain  of  teams 
into  a  limited  number  of  types  that  are  operationally  different. 

Differences  in  performance  among  teams  within  a  type  are  attributable 
to  differences  on  these  dimensional  attributes. 

Differences  in  team  types  are  important  only  if  there  are  associated 
differences  in  performance  or  training  requirements.  Differences  in 
performance  can  be  reflected  in  either  the  level  of  achievement  on 
specific  activities  or  the  capacity  to  do  some  things  but  not  others. 
Differences  in  training  requirements  can  be  in  either  the  knowledge  and 
skill  or  in  the  instructional  strategy. 

Differences  among  teams  can  be  characterized  in  at  least  two  ways: 

1)  the  type  of  application  or  operational  mission  they  perform,  and 

2)  variation  on  team  attributes  such  as  structure,  operational  processes, 

2 

and  composition.  Type  of  application  includes  the  C  functions  performed. 
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characteristics  of  the  tasks,  the  internal/ external  organizational  structure, 
and  the  input/output  links  with  external  organizations. 

Different  kinds  of  teams  can  be  defined  by  fixing  values  for  the  team 
attributes.  An  heuristic  device  for  focusing  on  relevant  team  structures 
would  be  to  describe  some  existing  teams.  For  example,  the  surveillance 
section  of  a  CRC  operates  by  dividing  the  airspace  into  regions  with  a 
technician  detecting  and  "picking"  tracks  in  his  region.  The  target  tracks 
are  passed  on  to  the  identification  section.  This  team  has  homogeneity 
of  functions,  medium  to  low  level  of  interdependence,  and  infrequent, 
medium-critical  coordination  and  communication.  In  contrast,  the  team 
of  weapons  director /technician  and  pilot  flying  an  intercept  has  a  high 
degree  of  heterogeneity  and  interdependence,  and  communication  is  a 
critical  area  of  behavior. 

Similarly,  operational  missions  of  teams  can  be  examined  to  determine 
if  they  can  be  categorized  into  distinct  types.  If  different  types  of  teams 
and  applications  can  be  identified,  the  degree  of  association  between  these 
two  sets  can  be  determined.  A  first  approximation  could  be  made  by 
attempting  to  compile  a  matrix  of  teams  x  application  in  order  to  highlight 
a  relationship.  Permuting  and  ordering  the  rows  and  columns,  for 
example,  might  reveal  a  pattern  of  contingency.  The  contingency  would 
then  permit  a  user  to  conclude  that  application  type  x  "requires"  team 
type  a.  If  the  properties  of  the  team  could  then  be  determined  by  a 
table-look-up,  the  systems  analyst  would  have  a  valuable  aid  in  addressing 
team  design  problems. 
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Thus  far,  we  have  identified  no  types  of  teams  or  applications  beyond  the 
level  of  some  preliminary  speculation.  The  problem  of  choosing  a  level 
of  discourse  is  an  initial  stumbling  block.  One  tends  to  shift  among 
different  molar /molecular  levels  and  some  consistency  needs  to  be 
imposed  before  reporting  these  ideas.  However,  some  of  this  speculation 
will  be  shared  with  the  reader  for  the  purpose  of  illustrating  the  thinking. 

Some  types  of  applications  are:  1)  intercept  direction  as  representative 
of  a  class  consisting  of  directing  or  vectoring  a  system  to  a  target  or 
other  objective,  2)  surveillance  for  and  tracking  of  potential  threats, 

3)  evaluating  and  targeting  in  accordance  with  an  operational  order  or 
plan  of  maneuver,  4)  deployment  of  sensor  systems,  5)  allocation  of 
resources  of  weapons  in  accordance  with  a  tactical  plan  and  current 
conditions,  6)  management  of  workload  and  throughput,  7)  anticipating 
and  adjusting  to  emergencies  or  unanticipated  conditions,  8)  real-time 
monitoring  and  evaluation  of  team  or  system  performance,  9)  managing 
external/internal  coordination,  and  so  on  until  one's  free-associative 
capacity  is  exhausted.  These  activities  consist  of  sets  of  tasks  which 
are  performed  at  least  in  part  by  collections  of  individuals  who  must 
act  in  concert  to  achieve  some  product,  objective,  or  goal. 

There  are  fewer  types  of  teams  to  suggest.  One  is  a  horizontal  structure 
consisting  of  individuals  who  perform  the  same  tasks  but  divide  up  the 
workload.  There  is  a  little  interdependence  and  little  differentiation 
among  members.  Surveillance  and  air  traffic  control  tend  to  be  of  this 
type;  the  workload  is  divided  by  geographic  area.  Another  way  of 
dividing  workload  is  to  take  items  on  a  basis  of  first  in  queue,  first 
out  to  the  first  available  person. 


A  team  that  has  its  members  highly  differentiated  in  function,  and  highly 
interdependent,  is  represented  by  the  combination  of  intercept  controller/ 
intercept  pilot.  There  are  variations  in  the  situation  of  one  controller 
directing  two  or  more  pilots  against  a  single  hostile  and  one  controller 
directing  several  one-on-one  intercepts  concurrently. 

A  hierarchical  structure  exists  in  supervision  and  management.  For 
example,  a  surveillance  or  weapons  supervisor  can  be  managing  several 
operators  or  sections  of  operators.  The  supervisors  may,  in  turn,  be 
under  the  direction  of  one  or  more  levels  of  battle  staff  management. 

Some  teams  process  things  in  a  serial  or  mixed  parallel /serial  order. 

The  sequence  of  activities  of  detection,  identification,  evaluation,  and 
action  selection  is  carried  out  by  a  team  arranged  in  this  manner.  There 
are  teams  within  each  of  those  activities. 

These  examples  are  speculations  which  we  intend  to  verify  and  extend 
when  an  opportunity  presents  itself.  How  to  get  them  into  a  matrix 
format  is  puzzling.  Better  definition  of  the  properties  and  uses  of  the 
matrix  may  be  the  next  step.  Also,  it  seems  relevant  to  consider  how 
one  determines  whether  a  pairing  of  teams  and  application  is  good, 
effective,  optimal,  or  some  other  evaluative  criteria. 
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CHAPTER  III 


RESEARCH  AREAS  AND  RECOMMENDED  LONG-TERM 
RESEARCH  TOPICS 


2  2 

Research  areas  in  C  T  will  be  discussed  for  the  purpose  of  establishing 
a  context  and  background  and  developing  specific  research  recommendations. 
The  discussion  is  intended  to  address  the  recommended  long-term  research 
topics  and  objectives  in  Table  7,  p.  54,  Volume  V  (Executive  Summary). 

The  topics  are  organized  into  six  categories  for  convenience  in  presentation. 
They  are  listed  in  Table  2  under  those  headings.  The  numbers  in 
parentheses  refer  to  the  issues  and  problem  areas  in  Table  4,  Volume  V. 

The  categories  are  used  only  for  convenience  in  organizing  the  text  to 
avoid  a  long,  unstructured  list  which  could  induce  deeper  levels  of 
boredom.  They  are  not  intended  to  be  definitive,  comprehensive,  nor 
exhaustive.  They  are  also  not  mutually  exclusive  and  other  partitionings 
of  the  topic  are  easily  devised. 

The  format  for  discussing  the  topic  areas  consists  of  a  brief  statement 
of  the  area,  discussion  of  problems  or  issues,  recommendations  of  a 
specific  project  intended  typically  as  a  starting  point,  and  an  outline  of 
the  approach  and  resources  needed.  They  consist  of  an  approach,  major 
steps,  types  of  resources  and  personnel  required,  and  a  level  of  effort. 
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TABLE  2.  LONG-TERM  RESEARCH  TOPICS 


•  Determination  of  deficiencies  in  performance  and  training 

for  rr2  (all) 

•  Develop  a  conceptual  model  and  framework  for  the  descriptions 

and  analysis  of  C2  team  functions  and  behaviors.  (1) 

9 

•  Methodology'  for  analysis  of  C"  team  behaviors: 

9 

--Develop  analytical  techniques  for  deriving  T“  requirements 
and  compile  supporting  data.  (3) 

--Develop  and  evaluate  systematic  procedures  for  determining 
the  skill  and  knowledge  requirements  of  simulated  combat 
missions  for  C2  teams  and  partition  them  over  the  categories 
of  individual,  preteam,  team,  and  superteam  exercises  as 
most  appropriate  vehicle  for  training.  (4) 

--Develop  and  evaluate  techniques  for  defining  the  required 
components,  functional  capabilities,  and  level  of  fidelity 
for  simulation  of  C2  teams  in  a  weapons  system.  (7) 

--Develop  techniques  for  interpreting  measures  of  team 
performance  to  identify  deficiencies  in  skill  and  knowledge 
and  to  select  appropriate  remedial  action. 

9 

•  Performance  quantification  and  measurement  of  C“  behaviors 

for  individuals,  teams,  and  systems: 

9  9 

--Develop  comprehensive  performance  objectives  for  C“T~ 
at  the  level  of  preteam,  team,  and  superteam  exercises.  (2) 

--Develop  and  evaluate  a  set  of  performance  measures  that  will 
assess  competence  of  C2  teams  and  battle  staffs  and  relate 
team  competence  to  system  effectiveness  and  individual 
proficiency.  (2) 

--Develop  techniques  for  analysis  and  interpretation  of  team 
performance  measures  in  terms  of  the  combat  readiness  of 
teams  and  systems.  (14) 

9 

•  Man-machine  design  for  C“  teams: 

--Several  of  the  topics  sorted  into  other  categories  are 
applicable  here  as  well,  especially  Methodology  and 
Performance  Quantification. 

--Conduct  research  to  evaluate  the  comparative  strengths 
and  weaknesses  of  hard  and  soft  C2  teams  on  performance 
of  critical  C2T2  tasks.  (18) 
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TABLE  2.  LONG-TERM  RESEARCH  TOPICS  (concluded) 


•  Development  and  implementation  of  training  requirements: 

2 

--Develop  a  set  of  standardized  C  team  exercises  which 
are  graded  in  difficulty  and  cover  the  range  of  C2T2 
operational  activities.  (2) 

--Identify  and  evaluate  existing  instructional  strategies 
applicable  to  T2.  (9) 

--Identify  new  topics  and  data  for  research  when  existing 
technology  does  not  provide  appropriate  or  adequate 
instructional  strategies.  (9) 

•  Simulation  for  C  exercises 

--Develop  data  for  simulation  of  operational  situations 
for  C2  teams  and  battle  staffs  which  identify  the  a) 
independently  controllable  features  of  the  simulation; 
b)  necessary  and  sufficient  conditions  of  fidelity  to 
reproduce  the  desired  behaviors;  and  c)  cost-benefit 
relationships  for  inclusion  of  each  feature  in  a  simulation.  (5) 

--Conduct  research  studies  on  the  comparative  cost-cost- 
effectiveness  of  live  and  simulated  combat  exercises- - 
and  mixes  of  these  events.  (16) 

•  Personnel  requirements: 

--Develop  and  assess  feasibility  of  a  psychometric  research 
plan  to  identify  selection  criteria  for  entry  into  a 
career  field  of  air  weapons  controller.  (8) 
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The  statement  of  approach  and  resources  is  more  global  than  is 
ultimately  needed  for  planning  and  implementing  a  research  program. 
However,  a  more  detailed  statement  requires  at  least  one  more  iteration 
of  the  recommended  projects.  That  iteration  would  provide  more  information 
on  the  specific  topic  or  sequence  of  topics  to  be  investigated,  the 
informational  objective,  study  design,  methods,  types  of  data  collection, 
and  amount  of  data  analysis.  Then,  man-hour  requirements  and  budget 
could  be  estimated.  This  information  is  not  available  at  this  time  nor 
are  there  resources  to  support  the  analysis. 

What  is  needed  at  this  stage  of  developing  a  research  program  is  the 
formulation  of  and  agreement  on  research  avenues  and  strategies. 

The  strategies  then  become  the  basis  for  more  detailed  planning  of 
studies.  Successful  formulation  of  the  strategies  may  be  possible  only 
in  working  sessions  where  the  options,  topics,  goals,  and  approaches 
are  "wrung  out.  " 

FURTHER  DETERMINATION  OF  DEFICIENCIES  IN  PERFORMANCE 
AND  TRAINING  FOR  C2T2 

An  early  major  step  in  a  research  program  should  be  a  systematic  study 

to  define  deficiencies  and  problems  in  team  performance  and  training 

more  precisely  and  in  more  detail.  This  further  definition  is  dependent 

2 

on  some  progress  in  developing  a  conceptual  framework  for  C  teams. 

It  will  provide  a  better  basis  for  formulating  queries,  identifying  appropriate 
behaviors  and  issues,  and  interpreting  data  and  observations.  The  survey 
in  the  present  study  provided  information  but  the  major  accomplishment 
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was  to  identify  problems  and  needs.  We  did  not  get  sufficiently  detailed 
information  on  conditions  to  analyze  the  problems  in  depth.  A  subsequent 
study  could  build  on  the  survey  and  develop  probes  to  provide  deeper 
analysis. 

The  potential  impact  of  these  deficiencies  on  system  effectiveness  can 
be  assessed  and  cost  estimates  could  be  made  for  specific  research 
studies  and  projects.  They  can  then  be  evaluated  on  the  basis  of 
cost- benefit  comparisons. 

The  most  objective  way  to  evaluate  command  team  performance  is  to 
assess  performance  on  an  exhaustive  set  of  terminal  behavioral  objectives 
for  team  and  operational  tasks.  The  most  practical  and  feasible  way 
to  assess  performance  is  to  use  technical  experts  as  raters  or  judges. 

The  performance  objectives  will  have  to  be  developed.  There  are  some 
existing  objectives  and  standards  from  the  STAN/EVAL  program,  for 
example,  which  can  be  collated  for  this  purpose.  Course  training 
standards  would  also  be  useful;  however,  they  must  be  evaluated  for 
adequacy  in  terms  of  coverage,  comprehensiveness,  metric  properties, 
interpretability.  and  appropriateness  for  the  research  objectives.  It  will 
probably  be  necessary  to  refine  and  supplement  them.  Team  behaviors  in 
particular  will  not  be  available. 

Such  a  project  would  take  three  years  or  more  to  produce  usable  results. 
Data  collection  would  have  to  take  place  in  operational  units  and  schools 
during  preteam,  team,  and  superteam  exercises.  Judges  will  have  to  be 
trained  and  massive  quantities  of  data  will  be  accumulated  for  analysis  and 
interpretation. 


The  complexity  of  the  effort  dictates  that  the  initial  project  be  a  study  of 
a  small  number  of  limited  situations  rather  than  a  broad  coverage  of  the 
entire  command.  Coverage  can  be  extended  in  subsequent  studies  for 
as  long  as  the  results  are  useful  for  the  purposes  of  research  planning. 

A  simulation  facility  for  command  and  control  teams,  similar  to  the  US 
Army's  Combined-Arms  Tactical  Training  Simulator  (CATTS),  would  be 
useful  in  evaluating  team  under  more  controlled  conditions.  However, 
development  of  the  facility  would  cost  several  million  dollars  and  be 
costly  to  support  and  operate.  Therefore,  it  must  be  a  longer-term 
objective. 

The  outcome  of  the  field  studies,  or  series  of  studies,  would  be  a  set 
of  research  needs  and  objectives.  They  will  be  a  prioritized  set  based 
on  objective,  empirical  performance  data,  and  on  evaluation  of  the  potential 
contribution  to  operational  effectiveness. 

DEVELOP  A  CONCEPTUAL  MODEL  AND  FRAMEWORK  FOR  C2  TEAM 
PERFORMANCE 

The  development  of  a  framework  for  team  phenomena  is  a  fundamental 
need  that  underlies  all  research  topics  and  would  be  highly  beneficial  to 
planning  an  integrated  program.  This  need  has  been  identified  by  others. 
For  example,  in  a  workshop  on  team  performance  held  by  RAND  under 
sponsorship  of  the  Office  of  Naval  Research  (ONR),  a  need  for  a  taxonomy 
was  identified  (Reference  4).  Although  a  taxonomy  is  not  a  model,  this 
statement  reflects  the  need  for  a  conceptual  framework  for  describing, 
discussing,  and  manipulating  team  phenomena. 
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The  term  framework  is  ambiguous.  We  have  chosen  it  to  mean  a  model 
in  the  sense  of  symbolic,  mathematical  representation  or  a  model  which 
can  be  represented  in  software  as  a  set  of  relationships  and  rules 
run  on  a  computer.  These  statements  are  objectives;  the  early  stages 
may  not  be  well-defined  or  may  have  little  mathematical  power.  However, 
the  formulation  should  be  mathematical  in  form. 

A  taxonomy  is  another  meaning  of  framework.  It  is  typically  a  set  of 
terms  that  can  be  used  for  sorting  and  labeling  things.  A  good  taxonomy 
would  provide  mutually  exclusive  categories  which  are  an  exhaustive 
partitioning  of  some  specified  domain.  More  sophisticated  taxonomies  have 
a  hierarchical  structure. 

Taxonomic  classification  would  be  an  early  stage  of  developing  a  model. 

The  model  would  also  represent  the  team  phenomena  and  processes  as 
a  functional  network  tnrough  which  information  is  processed.  The  model 
permits  the  user  to  insert  input  data  on  initial  conditions  and  calculate 
an  output  when  suitably  quantified  relationships  and  algorithms  are 
incorporated  into  the  model.  The  user  can  run  "what-if"  problems. 
Therefore,  we  opt  for  the  model  even  though  it  can  be  a  difficult  and 
complicated  undertaking. 

Developing  a  taxonomy  is  itself  a  difficult  undertaking  with  some  significant 
pitfalls.  Taxonomic  efforts  must  be  proposed  with  some  reservation  and 
concern  for  their  usefulness.  Their  history  is  less  than  satisfying. 

They  have  tended  to  provide  some  useful  tools  but  they  also  seem  to  depend 
on  arbitrary  rules,  to  be  incomplete,  and  to  absorb  unlimited  resources. 
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However,  some  functionally- oriented  taxonomic  structure  is  needed  in 
order  to  impose  some  order  on  this  problem  area.  It  is  not  feasible  to 
develop  a  classification  scheme  that  is  exhaustive  of  team  phenomena 
and  mutually  exclusive  in  its  categorization.  However,  it  should  be 
feasible  to  build  a  useful  taxonomy  for  a  defined  set  of  applications  within 
a  reasonable  cost. 

The  key  to  feasibility  and  tractability  of  a  taxonomy  is  to  identify 
explicitly  a  domain  of  applications  and  phenomena  of  teams  to  be  addressed. 

One  approach,  for  example,  might  undertake  to  describe  the  team  structure 
and  behaviors  in  a  CRC  or  a  TACC.  The  resulting  taxonomy  could  be 
extended  to  and  tested  by  application  to  related  team  situations,  such  as 
the  WOC.  Taxonomies  should  always  be  built  as  terminology  for  specific 
purposes.  The  comprehensive  and  general  approach  may  be  an  investigator's 
Holy  Grail,  which  may  consume  several  lifetimes  in  an  ineffectual  pursuit. 

The  other  side  of  the  issue  is  the  consequences  of  not  having  some  kind  of 
framework.  The  lack  of  a  systematic,  comprehensive  framework  for 
describing  and  analyzing  teams  and  human  factors  requirements  for  team 
operation  is  a  fundamental  deficiency  underlying  all  research  issues  on 
teams.  This  deficiency  is  manifested,  for  example,  in  a  lack  of  adequate 
terminology  and  taxonomies  for  describing  teams  and  behavioral  phenomena 
in  teams,  definition  of  the  types  of  activities  done  by  teams  and  team 
behaviors  in  performing  them,  definition  of  the  types  of  teams  that  occur 
in  military  systems,  and  differentiation  between  teams  and  multi- individual 
aggregates  of  people  working  independently.  They  are  a  necessary  precondition 
for  formulating  and  implementing  a  research  program  to  support  human 
factors  of  teams.  If  there  is  no  framework  which  can  serve  as  a  common 


reference,  it  is  not  possible  to  assess  what  is  currently  known  about  teams; 
it  is  difficult  to  compare  studies  and  integrate  the  results  because  even 
studies  differ  on  more  than  one  variable  and  processes.  The  conceptual 
framework  would  provide  a  means  for  establishing  equivalencies  and 
differences  among  related  studies,  for  identifying  relevant  variables  and 
relationships,  and  for  evaluating  the  adequacy  of  experimental  designs. 

( 

2 

Development  of  a  model  of  C  team  behaviors  is  proposed  as  a  basic 
first  step  in  a  research  program.  It  should  be  a  continuous  effort  with 
the  results  of  analyses  and  experiments  being  fed  back  into  the  model  to 
improve  its  accuracy.  The  model  will  go  through  iterations  in  accordance 
with  the  results  of  field  studies,  analysis,  computer  simulation,  and 
experimentation.  In  addition,  evaluation  and  feedback  can  be  obtained 
through  its  use  in  preparing  aids,  guidelines,  data,  and  methods  for 
use  by  developers,  designers,  and  managers  of  systems  and  training. 

The  following  section  contains  a  discussion  of  a  proposed  approach  to 
building  the  model.  The  immediate  objective  of  developing  the  model 
would  be  to  establish  the  nature  of  teams  in  four  ways: 

1.  Structure  of  teams  using  a  set  of  dimensions  or  attributes 
on  which  teams  can  vary  with  a  corresponding  variation 
in  team  performance. 

2 .  Processes  by  which  teams  accomplish  their  functions  and 
work. 

3.  Internal  and  external  variables  that  affect  the  processes 
and  performance  of  teams. 
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4.  Team  behaviors  and  skills  by  which  the  activities  of 
teams  can  be  described. 

This  information  is  needed  for  devising  a  taxonomic  and  functional 
network  and  would  also  be  useful  in  other  research  studies. 

2 

The  approach  to  developing  a  model  of  behaviors  of  C  teams  is  depicted 
in  Figure  1.  The  process  of  model  development  is  represented  inside  the 
dashed-line  box.  The  model  is  tested  by  experimentation  to  verify 
hypotheses  derived  from  the  model  or  interpretation  of  the  model  to  an 
application.  The  end  point  of  model  development  is  a  conceptual  framework 

9 

for  the  description  and  classification  of  C“  teams.  This  framework 

2 

presumes  that  the  model  includes  a  classification  of  types  of  C  applications 

2 

and  types  of  teams;  C  team  behaviors  are  identified  within  the  context 

2 

of  application  and  team  type.  An  inventory  of  C  team  behaviors  would 
be  obtained  by  taking  the  logical  union  of  these  classifications. 

There  are  several  caveats  to  be  made  concerning  Figure  1:  There  are 

many  feedback  links  and  interrelationships  not  shown.  It  is  not  a 

2 

structural,  functional  analysis  of  C  .  It  does  not  define  the  information 
to  be  gathered.  Rather,  it  is  a  representation  of  the  major  components 
of  an  approach;  it  has  an  input  level  at  the  top,  processing  in  the  center, 
and  output  at  the  bottom. 
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Approach  to  Development  of  a  Model  of  C  Team  Behaviors 


The  approach  is  tied  to  an  empirical  data  base  at  the  input  end  and  to 
technical  human  factors  areas  in  weapon  system  development  at  the  output 
end.  The  empirical  data  base  is  derived  from  four  sources: 

2 

•  Observation  of  C  teams 

2 

•  Observation  of  simulated  exercises  for  C  teams 

2 

•  Interviews  with  C  personnel 

•  Technical  literature 

The  approach  to  model  development  is  also  tied  to  human  factors  areas 

in  weapon  system  development.  Two  methodological  developments  are 

2 

needed  first:  a  methodology  for  analysis  of  the  behaviors  of  C  teams, 

2 

and  a  performance  quantification  of  the  behaviors  of  C  teams.  These 
methods  are  then  extended  to  the  design  and  support  of  weapon  systems. 
The  human  factors  objectives  and  technology  in  system  design  are  grouped 
into  five  areas: 

2 

•  Man- machine  design  for  C  systems /teams,  including  team/ 
man- computer  interface  requirements 

•  Personnel  requirements 

•  Performance  measurement  for  individuals  in  teams  and  systems 

•  Development  and  implementation  of  training  requirements 

2 

•  Simulation  for  C  exercises 
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Inadequate  attention  is  paid  to  team  behaviors  in  part  because  few  people 
know  what  they  are.  There  is  considerable  lore  about  techniques  for 
team  building.  Much  has  been  written  about  teams  and  team  behavior 
but  there  is  little  consensus  or  explicit  standardization  of  terminology, 
definition,  and  usage.  Terms  such  as  cooperation  and  communication 
are  recurringly  used,  but  it  is  very  difficult  at  best  to  interpret  them  as 
quantifiable  behaviors  that  can  be  stated  as  performance  objectives  and 
performance  measures. 

The  desirability  of  studying  teams  in  their  operational  environments  is 
discussed  in  the  previously- cited  RAND  report  (Reference  4).  This  is  because 
laboratory  research  on  small  groups  and  teams  has  had  little  applicability  to 
operational  problems  and  there  is  a  pressing  need  for  improved  team 
performance  and  methodology  of  team  training. 

We  agree  with  the  RAND  symposium  participants  that  studying  teams  in  situ 
is  necessary  but  also  maintain  that  it  is  not  sufficient.  The  position  has 
merit  in  that  it  forces  an  orientation  to  operational  tasks  and  conditions, 
ensuring  relevance  of  the  findings.  A  research  program  on  team  training 
should  start  with  observation  and  analysis  of  teams  in  operational  settings 
for  the  purpose  of  defining  the  domain  of  teams  to  which  one  wishes  to 
generalize  and  apply  the  findings  of  research.  Definition  of  the  domain 
would  be  in  terms  of  team  structures,  behaviors,  processes,  and  variables 
that  affect  these  things. 
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However,  research  in  operational  settings  is  often  both  very  costly  and 
impractical.  The  costs  arise  from  personnel  and  from  the  inefficient  use 
of  time  due  to  the  lower  priority  of  research  vs  operational  activities. 

The  impracticality  arises  from  several  sources:  limitations  on  the  control 
of  variables,  lack  of  freedom  to  manipulate  conditions  to  provide  an 
adequate  test  protocol,  and  inability  to  reconfigure  systems  and  procedures 
to  evaluate  alternatives.  Further,  ability  to  generalize  the  findings  is 
limited  because  two  operational  settings  may  be  no  more  alike  than  either 
is  to  a  laboratory  setting. 

The  consequence  of  these  shortcomings  is  that  one  observes  what  is 
available  to  observe  rather  than  what  should  be  observed  to  advance  the 
state  of  knowledge  and  develop  solutions  to  practical  problems. 

An  alternate  strategy  consists  of  starting  with  observation  in  operational 

settings  and  abstracting  from  them  to  construct  models,  simulations,  and 

experimental  conditions  that  are  necessary  and  sufficient  to  test  hypotheses. 

This  strategy  is  outlined  in  Figure  2.  The  figure  is  an  elaboration  and 

2 

rearrangement  of  the  upper  part  of  Figure  1.  Observation  of  a  C  application 

leads  to  identification  and  description  of  the  team  structure,  behavior, 

processes,  variables,  and  skills.  Interviews  and  analysis  of  the  technical 

literature  are  used  as  ancillary  techniques  to  facilitate  the  analysis  of  the 

observations.  It  is  postulated  that  several  application-specific  descriptions 

of  teams  can  be  collapsed  into  more  generic  descriptions  of  types  of 

teams  and  applications.  If  there  are  n  applications,  there  would  be  less 

than  n  types.  Modeling,  simulation,  and  experimentation  would  be  done 

within  the  context  of  these  types,  leading  to  improved  systems  for  description 

2 

and  classification  of  C  teams.  The  description/classifications  are  iteratively 
refined  through  application  and  evaluation  to  operational  settings. 
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Figure  2.  Strategy  for  Research  on  Command  and  Control  Teams 


The  second  approach,  abstracting  from  operational  settings  to  better- 
controLled  and  more  flexible  conditions  of  modeling  and  simulation,  is 
recommended.  Working  from  the  more  specific  operational  setting  to  a 
more  general  representation  in  the  laboratory  provides  anchors  in  the 

real  world  that  can  promote  representativeness  and  reality  in  the  analysis 
and  experimentation.  Further,  the  application  to  operational  settings 
in  the  form  of  solutions  to  operational  problems  and  system  design 
provides  an  evaluation  against  needs  and  conditions  of  the  real  world. 

The  indicated  scope  of  the  model- building  activity  is  much  too  Large 
to  be  taken  as  an  immediate,  as  opposed  to  an  ultimate,  objective.  The 
work  must  be  broken  into  phases  which  will  ultimately  provide  a 
comprehensive  analysis  of  the  Air  Force's  needs  in  a  technology  for  design 
and  training  of  teams.  The  program  should  be  initiated  with  an  analysis 
and  model  development  for  one  or  more  specific  situations,  such  as  a 
CRC  or  TACC. 

The  approach  for  the  initial  phase  would  consist  of  the  following  steps: 

2 

1.  Analyze  a  C  organization,  (CRC  or  TACC)  to  identify  and 
describe  team  phenomena,  particularly  those  involving  teams 
in  processing  information  and  decision-making.  These 
activities  should  be  described  for  representative  segments  of  an 
operational  cycle  during  a  combat  scenario.  The  descriptions 
should  include  the  type  of  task/function,  how  the  processing/ 

decision-making  is  done,  data  required,  and  the  form  of  output. 
The  structures  of  teams  in  the  organization  and  the  interactions 

among  team  members  should  be  identified. 
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2.  Review  existing  concepts  and  data  applicable  to  C  functions, 

man- machine  interfaces,  tasks,  and  team  performance  to  identify 

relevant  concepts,  terminology,  decision  processes,  variables, 

and  nature  of  the  relationship  between  variables.  The  purpose 

of  this  step  is  to  provide  systematic  terminology  for  analyzing 
2 

the  C  team  activities.  This  step  should  be  concurrent  with  and 
iterative  with  Step  1  in  order  to  get  a  synergistic  interplay 
between  concepts  and  observations. 

2 

3.  Formulate  a  conceptual  framework  for  C  team  activities  and 
processes.  This  framework  is  the  best  statement  one  can  make 
at  that  time. 

4.  Test  and  evaluate  the  framework  by  applying  it  to  the  description 

2 

and  analysis  of  specific  C  team  applications.  Generate  hypotheses 
about  command  team  behaviors  and  performance  and  test  by 
experimentation  and  observation. 

5.  Identify  deficiencies  in  the  framework  and  revise  as  appropriate. 
Iterate  cycle  of  apply,  prediction,  test,  and  revision. 

The  resources  needed  to  support  this  effort  are  difficult  to  estimate.  It 
is  a  long-term  project  with  several  intermediate  products  that  can  be  useful 
in  their  own  right. 

The  approach  to  estimation  is  to  develop  a  strategy  for  model  building 
with  long-term,  multi-year  objectives  and  a  sequence  of  intermediate 
objectives.  Specific  projects  can  be  defined  for  each  objective  and  the 
level  of  effort  estimated  for  those  projects. 
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Development  of  the  long-range  strategy  is  probably  a  one  to  two  man-year 
level  of  effort.  However,  extensive  interaction  is  necessary  among  the 
strategy  developer,  sponsor,  proponents,  and  potential  users.  This  kind 
of  effort  depends  heavily  on  creativity  as  well  as  observation  and  analysis. 
The  difficulty  of  scheduling  insights  and  breakthroughs  generates  problems 
for  planning  and  management. 

Precursors  of  a  Model  for  Command  Teams 


Some  ideas  for  a  model  grew  out  of  our  research.  Our  need  to  establish 
a  working  framework  for  planning  the  research  was  a  stimulus  to  this 
thinking.  We  will  present  these  ideas  as  a  starting  point  or  first 
approximation  to  a  team  model.  The  level  of  development  of  the  model  at 
this  point  is  primarily  a  basic  framework  that  can  be  elaborated  into  a 
detailed  description  of  team  operations  and  processes. 

2 

The  C  team  model  consists  of  four  principal  modules; 

1.  Team  module 

2.  Tactical  situation  model 

3.  Command  team  behavioral  module 

4.  Driver  scenario  module 

A  preliminary  conceptual  structure  has  been  provided  for  each  module. 
They  need  to  be  expanded  and  differentiated  in  more  detail  by  a  top-down 
analysis  and  application  to  one  or  more  specific  situations. 
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The  Application  Situation--We  will  start  with  an  unspecified  team  in  a 
tactical  combat  situation  as  part  of  a  Blue  force.  The  Blue  force  is 
opposed  by  a  Red  force,  which  is  executing  a  tactical  plan.  Red's 
tactical  plan  is  a  scenario  that  serves  as  an  external  driver  of  events 
impinging  on  the  Blue  team. 

The  Blue  force  has  been  tasked  with  an  operational  objective.  A  plan 
of  battle  has  been  developed  and  the  tasking  has  been  broken  down  into 
more  specific  tasking  for  subordinate  units.  The  team  being  modeled 
has  been  given  its  specific  tasking. 

When  a  military  command,  such  as  a  numbered  air  force,  wing,  or 
squadron,  is  given  an  operational  tasking  or  mission,  it  is  given  an 
objective  to  reach  or  hold,  a  threat  to  counter,  resources  to  allocate, 
and  assets  to  protect.  An  Air  Force  objective  broadly  is  control  of 
specific  airspace  or  strike  against  a  target  complex.  The  assets  have 
a  value  reflected  in  priorities. 

A  tactical  plan  must  be  prepared  and  implemented.  It  consists  of: 

1.  Deployment  of  forces  to  counter  threat 

2.  Allocation  of  resources  for  the  deployed  forces  to  expend 
against  the  threat 

3.  A  scheme  of  maneuver  to  protect  assets 

The  activities  of  the  command  team  are  organized  to  support  the  preparation 
and  implementation  of  this  plan.  The  acquisition  and  evaluation  of 
intelligence  and  the  logistics  to  support  anticipated  operations  are  examples 
of  activities  of  analysis  and  planning,  which  are  part  of  developing  the 
tactical  plan. 
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Team  vIodule--A  team  can  be  described  in  terms  of  two  sets  of  properties 
or  attributes :  architectural  dimension  and  functional  properties.  The 
architecture  dimensions  are  attributes  of  group  structure,  operational 
processes,  and  composition.  These  properties  are  determined  by  the 
operational  tasks  and  organizational  structure.  Functional  properties 
are  the  competencies,  skills,  and  knowledge  required  for  team  operation; 
they  are  the  team  skills.  They  are  determined  by  the  shared  plan  or 
schema.  Tables  3  and  4  contain  a  listing  of  these  properties.  Tbe  items 
in  each  list  are  derived  mainly  from  the  research  literature  and  are  a 
collation  of  attributes  attributed  to  teams.  The  principal  criterion  used 
in  compiling  the  lists  was  comprehensiveness;  the  items  are  neither 
mutually  exclusive  nor  well-defined  at  this  time. 

The  lists  represent  hypothesized  candidate  attributes.  Each  item  is  a 
potential  research  variable.  To  be  useful  each  item  should  be  related  to 
observable,  quantitative  measures  of  team  process  or  product. 

The  operational  mission  determines  primarily  the  architectural  attributes 
of  the  team.  The  functions  required  to  perform  the  mission  break  down 
into  tasks  and  behaviors  of  a  command  group  and  determine  the  subs  tan  tative 
content  of  the  tasks. 
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TABLE  3.  TEAM  ARCHITECTURAL  DIMENSIONS 


Structure: 

•  Assignment  of  individuals  to  roles 

•  Type  of  network 

•  Lateral- vertical  hierarchy  configuration 

•  Allocation  of  authority,  responsibility,  or  control: 
centralized  or  distributed 

•  Rigidity  vs  flexibility  of  the  relationships  between  roles 

•  Geographical  distribution  of  team  members:  dense  vs 
diffuse;  adjacent  vs  remote 

•  Internal  vs  external  determination  of  structure 

Operational  Processes: 

•  Goal,  objective,  or  purpose 

•  Product  or  output 

•  Partitioning  of  functions,  responsibilities,  activities, 
tasks 

•  Structure  of  subgoals  or  intermediate  products  as  indicators 
of  team  process 

•  Roles  of  operation,  managing,  decision-making 

•  Proportion  of  procedural  and  non- procedural  activities 

•  Interdependence  of  functions 

•  Degree  of  social/ emotional  support  to  members 
Composition: 

•  Number  of  members 

•  Types  of  specialty  codes 

•  Variability  of  competence  of  team  members 

•  Redundancy  of  overlap  of  functions 

•  Turnover,  rate,  continuity,  or  stability  over  time 

•  Compatibility  among  team  members 

•  Stability  of  assignment 

•  Personal  visibility 
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TABLE  4.  FUNCTIONAL  PROPERTIES  OF  TEAM 


Coordination- -what,  how,  timing 
Interaction-  -  interdependence 
Cooperation 

Compensatory  adjustment  among  team  members 

AbiLity  to  modify  actions  in  accordance  with  team  goals  and 
progress  to  them 

Adjusting  one's  performance  to  work  flow  within  team  network 

Improvization  of  impromptu  and  ad  hoc  responses  in  relation 
to  team  goals  and  other  team  functions 

Knowledge  of  the  team  operation  and  interdependence 

Knowledge  of  relationship  between  or  contribution  to  team 
goals  from  one's  own  function  or  role 

Mechanism  to  detect  and  correct  errors 

Adjustment  mechanisms 

•  Emergent  mechanisms 

•  Unexpected  conditions 

•  Emergencies 

•  Overloads 

Cue  and  alerting  to  change  in  condition 
Anticipatory  change  in  conditions 

Discipline  in  communication 

•  Relevance 

•  Appropriateness 

•  Timeliness 

Allocation  of  resources  and  workload 
Management  of  workload  and  overload 
Team  memory,  data  storage,  recordkeeping 


The  operational  tasking  determines  the  processes  necessary  for  the  group 
to  react  to  the  external  event  and  produce  an  output.  These  processes 
determine  the  number  of  team  members,  the  kinds  of  people  (AFSC), 
the  role  of  each  team  member  in  terms  of  the  duties  and  tasks  assigned 
to  him,  and  the  connections  between  members.  These  components  thus 
determine  the  size,  composition,  network  structure,  functional  heterogeneity, 
and  interdependence  of  the  team. 

The  functional  properties  in  Table  4  are  attributes  describing  how  well 
the  team  performs  or  the  quality  of  team  performance  rather  than  what 
the  team  does  or  what  it  accomplishes  in  output.  These  process  attributes 
are  of  two  kinds:  those  associated  with  the  tasks  of  the  operational  mission 
and  those  devoted  to  management  and  control  of  the  team  process.  Alloca¬ 
tion  and  adjustment  of  workload  is  one  of  the  latter,  for  example. 

The  attributes  are  determined  by  a  shared  plan  or  schema,  which  is 
distinct  from  a  tactical  plan.  The  shared  schema  is  the  team's  usually 
unspoken  agreement  on  how  they  will  interact  to  accomplish  the  operational 
mission.  Each  team  member  must  understand  the  schema,  what  it  requires 
of  him  in  terms  of  control  and  adjustment  of  his  behavior  to  coordinate 
with  other  team  members,  and  what  adjustments  other  team  members 
must  make  with  respect  to  each  other.  The  existence  of  a  team  entity 
and  the  quality  of  team  performance  are  an  increasing  function  of  the  level 
of  understanding  and  participation  in  the  shared  plan.  The  relationship 
between  team  quality  and  amount  of  output,  quantitative  or  product  measure, 
is  unknown. 
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Tactical  Module--The  focus  of  the  discussion  until  now  has  been  definition 

and  characterization  of  the  term  "team.  "  Another  key  aspect  a  C  team 

performance  model  must  deal  with  is  the  tactical  situation.  A  tactical 

command  operates  to  exert  force  against  an  adversary  force.  We  will  not 

get  into  military  theory  and  doctrine  for  combat  or  use  of  resources,  although 

2 

they  are  a  body  of  principles  that  shape  and  constrain  C  team  behavior. 

Only  defensive  and  offensive  situations  will  be  treated  here,  leaving  further 
analysis  until  later  iterations  of  the  model  development. 

The  tactical  module  of  the  model  represents  the  goal  of  a  tactical  plan 
as  establishing  and  maintaining  a  steady  state  of  balance  between  opposing 
forces.  The  steady  state  may  be  a  non-zero  value.  The  Blue  command 
seeks  to  maintain  it.  The  external  driver  scenario  produces  disturbers 
in  the  form  of  enemy  action  which  tend  to  upset  the  steady  state. 

When  cast  in  this  form,  the  tactical  module  becomes  a  control  module  which 
has  the  objective  of  controlling  and  modifying  the  process  of  a  system 
to  keep  a  state  or  output  within  some  specified  bounds.  The  output  is 
often  defined  as  an  optimum  for  some  functions.  This  type  of  problem  is 
in  the  domain  of  control  theory,  a  mature  area  of  technology.  There  are 
standard  methods,  techniques,  and  models  within  existing  control  theory. 

They  should  be  useful  for  this  application  and  should  be  reviewed  for  the 
purpose  of  assessing  the  feasibility  and  value  of  mapping  the  tactical 
module  into  some  available  model. 
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The  information  the  command  team  receives  is  incomplete  and  unreliable. 

The  command  team  is  attempting  to  determine  the  state  of  the  world  at 
some  remote  point  in  terms  of  threat  to  the  tactical  plan  and  to  implement 
counteraction  at  that  remote  point.  A  functional  breakdown  of  these  processes 
is  represented  in  Figure  3. 

2 

The  information  the  C  team  receives  at  a  TACC,  for  example,  is  in 
the  form  of  alphanumeric  messages.  The  messages  contain  data  about 
battlefield  and  airspace  events.  The  data  is  received  sequentially  and 
without  indicated  correlation  between  related  events.  The  data  is  only  a 
sample  of  the  information  in  the  situation  and  it  is  thus  incomplete.  It 
contains  inaccuracies  which  must  be  identified  and  removed  by  cross- correlation 
and  internal  consistency  among  sets  of  data.  This  data  must  be  used  to 
construct  the  events  of  enemy  action  that  have  occurred.  This  tactical 
picture  must  be  interpreted  in  terms  of  the  enemy's  intentions  and  possible 
actions. 

Another  area  of  activity  of  a  command  and  control  team  in  development 
and  implementation  of  the  tactical  plan  consists  of  two  phases:  force 
generation  and  engagement.  A  summary  of  these  phases  and  their 
characteristics  is  presented  in  Figure  4,  adapted  from  Wohl  (Reference  9). 

Force  generation  is  the  preparation  of  the  scheme  of  maneuver  and  the 
initial  allocation  and  deployment  of  resources  and  forces.  Engagement 
is  the  control  of  forces  in  response  to  enemy  action  and  execution  of  the 
scheme  of  maneuver.  These  two  phases  involve  functions  and  tasks  such 
as  assessment  of  threat,  evaluation  of  alternatives,  correlation  of 
intelligence  information,  and  preparation  of  operational  orders  at  a  more 
detailed  level. 
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ACQUISITION  Of  IMf  ORMATIOM  BASE 


Figure  3.  Functional  Areas  of  Command  and  Control 
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Figure  4.  Summary  of  Critical  Tactical  Command  and  Control 
Decisions  (adapted  from  Reference  9) 


2 

The  C  tear'  operates  in  two  phases:  planning  and  current  operations 
corresponding  to  force  generation  and  engagement.  The  team’s  function 
is  to  implement  a  tactical  plan  that  establishes  the  steady  state  and  monitor 
it  to  detect  and  nullify  disturbers.  The  team  can  respond  in  one  of  two 
modes:  reactive,  in  which  the  team  detects  and  fashions  responses  in 
real  time;  and  anticipatory,  in  which  the  team  identifies  possible  or 
probable  disturbers  during  planning,  establishes  a  search  routine,  and 
preplans  the  response.  Planning  and  implementation  can  be  intermixed 
temporally. 

The  objective  of  the  team  in  a  defensive  situation  is  to  keep  the  forces 
in  balance  of  equal  force  or  null  the  steady  state. 

The  steady  state  is  established  by  evaluating  Red's  capabilities  and 
intentions  and  countering  them  by  depolying  forces  against  them.  The 
factors  considered  in  deployment  are  control  of  critical  airspace,  areas 
of  surveillance,  assignment  of  sectors  for  patrol,  nature  of  the  threat, 
and  other  factors  of  tactical  value. 

The  function  of  the  team  in  attack  is  to  develop  and  implement  a  tactical 
plan  which  maintains  a  positive  steady  state  to  provide  the  required 
momentum.  In  an  attack  mode  the  steady  state  would  be  maintained  with 
a  positive  force  ratio  to  give  a  momentum  to  the  combat  element.  There 
would  also  be  a  sequence  of  positions  or  postures  which  are  sub-objectives 
or  subgoals  en  route  to  the  objective  of  the  attack.  They  probably  have 
associated  times  by  which  to  reach  them  as  in  the  concept  of  phase  lines. 
They  must  also  detect  and  neutralize  disturbers. 


59 


Command  Team  Behavioral  Module--The  next  steps  in  developing  the 

2 

model  are  to  develop  a  module  for  C  and  team  behaviors  on  the  skeleton. 

The  module  would  consist  of  the  functions  and  tasks  performed  by  members 

2 

of  a  command  team.  The  content  would  be  predominantly  the  C  activities 
in  operational  tasks.  The  activities  will  contain  functions  and  tasks 
performed  by  individuals,  multi- individual  aggregates,  and  teams.  Several 
teams  may  exist  and  the  team  composition  and  configuration  may  be 
different  for  different  functions.  These  characteristics  will  be  determined 
during  the  particularization  of  the  model  to  specific  applications. 

Our  level  of  development  of  this  module  is  embryonic.  We  know  there 

2 

must  be  one  for  the  purposes  of  research  on  manned  C  systems  and  we 
have  some  preliminary  ideas.  Some  of  the  material  presented  on  the 
tactical  module  belongs  here  but  we  have  not  yet  sorted  it  out.  The 
concepts  in  the  tactical  module  may  have  dual  facets;  one  facet  is  the 
tactical  function  and  the  other  facet  is  the  behaviors  of  the  operators. 

2 

Adequate  descriptions  of  C  behaviors  are  not  readily  available.  Performance 

objectives  are  not  available  for  command  teams.  Some  work  does  exist, 

2 

though,  which  can  serve  as  a  prototype  or  paradigm  for  defining  C  team 
behaviors.  Table  5  contains  a  list  of  12  tasks  from  a  simulation  for  Army 
battalion  command  teams  (Reference  10).  The  first  four  correspond 
roughly  to  the  pre-engagement  phase  and  the  remaining  eight  correspond 
to  the  engagement  phase.  These  tasks  are  further  broken  down  into  more 
detailed  behavioral  elements.  The  Army  has  had  some  experience  in  using 
this  task  list  and  the  simulation  for  training  and  research  in  the  CATTS. 

This  work  can  be  used  to  provide  guidelines  and  lessons  learned  in  identifying 
team  behaviors  and  developing  performance  objectives  and  standards. 
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Engagement  Force  Generation 


TABLE  5.  TASKS  FOR  BATTALION  COMMAND  GROUPS  FROM 
ARMY  TRAINING  EVALUATION  PROGRAM 


Task  1. 
Task  2. 

Task  3. 

Task  4. 

Task  5. 

Task  6. 

Task  7. 

Task  8. 
Task  9. 

Task  10 
Task  11 
Task  12 


Develop  plan  based  on  mission 

Initiate  intelligence  preparation 
of  the  battlefield 

Prepare  and  organize  the 
battlefield 

Troop  lead 


See  the  battlefield  during 
the  battle 

Control  and  coordinate  combat 
operations 

Employ  fires  and  other  combat 
support  assets 

Concentrate /shift  combat  power 

Manage  combat  service  support 
assets 

Secure  and  protect  the  task  force 

.  Troop  lead  during  battle 

.  React  to  situations  requiring 
special  actions 
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Sources  with  which  to  begin  developing  descriptions  of  AFC  tasks  are 
existing  task  inventories  cited  in  Volume  I,  Chapter  II;  TACM  50-9, 

Training  Tactical  Air  Control  System  (TACS),  1980;  STAN/EVAL 
behavioral  checklists;  and  the  scenarios  and  objectives  for  Blue  Flag 
exercises.  The  team  behavioral  module  can  be  best  developed  by  using 
the  existing  literature  and  observation  exercises  to  postulate  further 
details  of  the  modules. 

Driver  Scenario  Module- -A  driver  scenario  must  be  provided  to  generate 
the  tactical  events  to  which  the  Blue  Command  team  responds.  Again, 
our  thinking  is  still  at  the  embryo  stage.  The  scenario  must  include 
realistic  threat  levels,  tactics,  movement  constraints,  and  other  components 
of  a  conflict.  It  is  desirable  that  the  Red  force  be  an  intelligent  adversary 
who  has  options  for  varying  his  responses  and  plans  in  accordance  with 
emerging  tactical  conditions.  However,  the  complexity  and  cost  of  the 
scenario  module  increases  with  the  level  of  intelligence  and  flexibility; 
practicality  may  dictate  that  the  scenario  be  restricted  in  these  attributes. 

METHODOLOGY  OF  ANALYSIS  OF  C2  TEAM  BEHAVIORS 

This  area  of  research  is  tool  development  for  designers,  developers,  and 
2 

managers  of  C  systems.  The  methodology  must  be  usable  in  system 
development,  management  of  resources  during  deployment,  and  maintenance 
of  combat  readiness.  It  must  be  applicable  to  weapon  system  acquisition 
during  conceptual,  advanced,  and  engineering  development  and  to  the  design 
of  man-machine  interfaces,  workspaces,  personnel  subsystems,  training 
and  training  devices. 
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It  is  recommended  that  a  research  project  be  initiated  with  the  objective 

2 

of  developing  analytical  tools  appropriate  to  the  design  of  teams  and  C  . 

The  methods  must  permit  the  user  to  identify  the  behavioral  data  needed 

for  these  applications  and  select  or  generate  that  information  from 

engineering  design  data,  statements  of  required  operational  capability, 

or  mission  requirements.  The  methods  would  be  an  extension  of  current 

techniques  of  task  analysis  and  compatible  with  Instructional  Systems 

Development.  The  sequence  from  MIL-H-468552  (Reference  11)  should 

be  maintained.  It  establishes  an  analytical  procedure  consisting  of  the 

steps  of  defining  and  allocating  system  functions,  information  flow  and 

processing  analysis,  estimation  of  potential  operator /maintainer  processing 

capability,  allocation  of  functions,  analysis  of  tasks,  loading  analysis  and 

design  of  system  and  subsystems.  A  human  engineering  program  plan 

2 

and  a  test  and  evaluation  plan  are  also  required.  The  application  to  C 
teams  will  require  that  design  of  individual  work  stations  and  man-machine 
interfaces  be  supplemented  with  design  of  person-to-person  interfaces 
within  the  configuration  of  the  team  itself. 

The  data  to  be  extracted  is  determined  by  the  uses  to  which  it  will  be  put 
in  design  and  support  of  systems.  Generation  of  only  task- analytic  data 
is  not  enough.  The  user  must  also  be  provided  with  techniques  and  algorithms 
for  processing  the  task  data  to  provide  solutions  to  the  problems  of 
design  of  team  structures,  man- machine  interfaces,  personnel  requirements, 
and  training. 
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There  has  been  recent  work  oriented  in  this  direction.  It  has  mostly 
2 

dealt  with  C  systems  and  emergent  situations  involving  semi- structured  tasks. 

For  example,  guidelines  for  requirements  definition  and  design  of  man- 

2 

machine  interfaces  in  C  systems  have  been  prepared  by  Smith  (Reference 
12)  and  by  Ramsey  and  Atwood  (Reference  13)  for  man-computer  systems. 

The  reports  contain  excellent  summaries  of  design  options  and  illustrations 
as  well  as  the  techniques  for  performing  the  analysis.  Another  relevant 
effort  is  the  review  of  performance  models  applicable  to  evaluation  of 
man-machine  systems  (Reference  14).  The  report  is  a  compendium  of 
descriptive  information  on  the  models  with  evaluative  analysis.  Research 
recommendations  are  also  provided. 

Two  conditions  are  necessary  for  the  effective  use  of  the  task  analytic 
tools:  Data  item  descriptions  (DIDs)  must  be  prepared  to  specify  the 
format  and  content  of  the  task  data  to  be  provided  and  the  task  data  must 
be  specified  as  deliverable  items  under  a  procurement  contract  for  a 
system.  Contractual  obligation  to  deliver  the  data  will  ensure  its 
availability  in  useful  form. 

A  training  analysis  requires  more  detailed  breakdown  of  task  descriptions 
in  order  to  provide  the  level  of  detail  needed  to  determine  training 
requirements  and  develop  or  procure  training  courses  and  equipment.  The 
skills  and  knowledge  required  for  competence  of  the  team  and  tasks 
behaviors  should  be  determined  and  translated  into  terminal  and  enabling 
performance  objectives.  The  levels  of  competence  should  be  defined  in 
terms  of  levels  of  mastery  associated  by  skill  levels  of  the  Air  Force 
Specialty  Code  (AFSC).  These  levels  entail  a  partitioning  of  behaviors 
among  skill  levels  within  grades  of  a  specialty.  These  analyses  should  be 
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an  extension  of,  based  on,  or  compatible  with  the  procedures  and  regulations 
for  Instructional  System  Development,  which  has  become  a  standardized 
practice  in  the  training  community. 

3 

The  development  of  techniques  for  task  and  training  analysis  for  C  teams 

3 

will  have  to  confront  the  challenge  that  the  operational  tasks  for  C  teams 
are  semi-structured  rather  than  proceduralized  tasks  and  are  performed 
in  emergent  situations.  Conventional  task  analysis  is  best  suited  to 
proceduralized  tasks  and  cannot  adequately  handle  the  contingencies  and 
dependencies  of  the  information  flow  in  emergent  situations.  The 
variability  of  events  that  can  occur  may  require  analysis  in  terms  of 
functions,  content,  and  sets  of  equivalent  behaviors  rather  than  fixed 
sequences  of  responses.  The  problems  entailed  are  similar  to  those 
encountered  in  computer-assisted  instruction  (CAI)  authoring  languages 
and  the  design  of  man- computer  dialogues. 

Task  analysis  is  a  very  labor-intensive  undertaking  even  for  the  more 
tractable  analysis  of  procedural  tasks.  This  condition  will  become 
much  more  serious  for  the  semi- structured  tasks.  Tasks  will  have 
to  be  represented  as  networks  or  tree  structures  in  which  task  options 
are  represented  as  alternative  behavioral  chains  and  the  situational 
factors  on  which  the  branching  among  them  depends  are  identified  and 
associated  with  each  chain. 

Data  processing  aids  for  the  analysis  of  information  flow,  function  allocation, 
and  task  description  will  be  needed  to  reduce  the  manpower  requirements 
and  increase  the  cognitive  scope  of  the  analyst  to  comprehend  the  structure 
of  the  tasks.  Such  aids  will  increase  the  feasibility  of  doing  these  analyses 
while  reducing  cost  and  increasing  effectiveness. 
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The  magnitude  of  an  effort  to  develop  these  techniques  and  the  investment 
required  is  often  a  deterrent  to  planners,  and  raises  difficult  problems 
of  priorities.  There  is  often  a  search  for  quick- and- dirty,  low-cost 
techniques  that,  though  inaccurate,  are  precise  enough  for  most  applications. 
However,  these  techniques  are  risky  because  the  analyses  are  difficult 
to  replicate  and  their  limitations  in  terms  of  errors  and  their  consequences 
are  unknown. 

The  approach  to  develop  these  methods  should  consist  of  three  major 
phases: 

1.  Identify  the  extended  procedure  or  sequence  of  analytical 

issues  and  steps  necessary  to  go  from  an  operational  need 

or  system  concept  to  completion  of  development  of  the  man- 

2 

machine  components  for  C  teams  such  as  operating  procedures, 

interfaces,  personnel,  training,  and  simulation.  Provide 

techniques  for  implementing  MIL-H-46855  and  define  DIDs  for 

2 

each  of  the  man-machine  components  of  C  teams. 

2.  Develop  or  compile  the  data  bases,  algorithms,  models,  data 
processing  aids,  and  procedures  for  implementing  the  systems 
and  task  analysis. 

3.  Shorten  the  extended  procedure  by  developing  more  efficient 
and  powerful  techniques  or  generating  new  knowledge  through 
research  on  command  teams  ‘hat  permit  simplifying  and 
shortening  the  procedure. 
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Developing  these  procedures  should  draw  on  concepts  and  techniques  for 
analysis  and  solution  that  are  available  in  operations  research,  systems 
analysis,  and  management  information  systems.  Task  analysis  entails 
problems  of  determination  of  requirements,  allocation  of  resources, 
analysis  of  work  flow,  and  evaluation  of  outcomes  that  are  standard 
procedures  in  these  disciplines.  It  is  very  probable  that  existing 
techniques  can  be  readily  adopted  to  the  task  analysis.  In  addition, 
computer  models  and  simulations  should  be  examined  for  adaptability 
to  human  factors  planning  and  incorporation  to  the  task  analysis  procedure. 

The  facilities  required  to  develop  these  techniques  are  principally  data 
processing  equipment  to  implement  the  procedures  as  developed.  Tryout 
and  evaluation  by  users  among  the  designers  and  developers  of  systems 
and  training  will  be  needed. 

The  work  should  be  done  both  in-house  and  on- contract.  Mixes  of 
specialized  skills  will  be  needed  but  will  vary  in  composition  from  project 
to  project.  The  best  way  to  get  this  flexibility  is  contracting  for  the 
short-term  availability  as  needed. 

PERFORMANCE  QUANTIFICATION  FOR  C2  TEAMS 

There  is  a  near-complete  lack  of  adequate,  objective  criteria  and  standards 
for  evaluating  team-oriented  skills,  individual  and  team  readiness,  and 
system  effectiveness  in  C2.  Two  types  of  evaluations  appear  to  be  used. 
One  is  an  observer's  judgment  or  rating  of  performance,  usually  on  an 
individual  basis  rather  than  team  or  unit.  The  other  type  of  measure  is 
completion  of  a  specified  number  of  actions  in  some  time  period,  such  as 


direct  x  number  of  Eve  intercepts  during  a  month,  quarter,  or  other  time 
interval.  Systematic  records  are  kept  at  squadron  and  wing.  However, 
the  entire  approach  to  performance  evaluation  is  neither  objective  nor 
well- controlled  and  gives  little  information  directly  interpretable  in  terms 
of  the  real  operational  effectiveness  and  training  needs  of  individuals, 
teams,  or  units. 

The  primary  research  topic  is  development  of  performance  measurement 
techniques  for  teams.  It  involves  complex  psychometric  issues  and 
information  about  the  system  application  in  a  tactical  situation.  In 
addition,  the  measurement  of  team  performance  should  be  done  in  the 
context  of  system  and  individual  performance  and  should  be  consistent 
with  them. 

The  desired  characteristics  of  a  performance  measurement  system  are 
discussed  in  the  following  section.  The  discussion  is  organized  into  the 
topics  of  product/process  measures  and  predictive /diagnostic  measures 
for  system,  team,  and  individual  performance. 

Product /Process  Measures 


Two  kinds  of  measures  of  the  competence  of  a  team  should  be  developed: 
1)  product  or  output  process  or  2)  process  measures. 
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Product  or  output  measures  are  the  observabLe  or  computable  indexes 
in  terms  of  numbers  such  as  intercepts  completed,  targets  detected, 
attrition  imposed,  etc.  They  are  related  to  system  or  mission  measures 
of  product.  Process  or  procedural  measures  reflect  goodness  of  the 
processes  used  in  accomplishing  tasks  and  duties.  Process  measures 
should  assess  performance  in  terms  of  appropriateness  of  the  action 
taken  relative  to  the  information  available  to  the  actor. 

Process  measures  have  a  connotation  of  conforming  to  protocol  or  a  fixed 
procedure.  This  property  is  in  apparent  conflict  with  the  semi-structured 
nature  of  tasks  in  emergent  situations.  However,  there  may  be  a  working 
solution  to  the  conflict.  It  is  hypothesized  that  the  incumbents  in  semi- 
structured  and  unstructured  situations  evolve  some  established  procedures 
and  methods.  These  procedures  are  standard  or  typical  ways  of  proceeding 
and  strategies  for  dealing  with  the  tasks  at  hand.  They  are  applicable 
to  the  representative  conditions  that  arise.  These  procedures  evolve  usually 
to  make  efficient  use  of  time,  minimize  errors,  minimize  risks,  and 
increase  the  likelihood  of  success. 

The  team  member  at  each  position  must  know  not  only  these  procedures  but 
the  conditions  under  which  and  how  to  deviate  from  it.  Process  measures 
should  then  assess  ability  to  use  the  procedures  appropriately,  recognize 
circumstances  under  which  they  are  not  appropriate,  and  adjust  to  an 
appropriate  measure. 
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Measures  of  team  performance  must  differentiate  between  those  behaviors 
related  to  functioning  within  the  team  and  those  related  to  the  operational 
tasks.  The  two  sets  of  measures  should  be  kept  separate  because  they 
represent  independent  domains  of  team  performance. 

The  team  behaviors  give  process  measures  reflecting  the  degree  of 
appropriate  interaction,  interdependence,  and  coordination.  There  are 
also  product  measures  for  teams:  equalization  of  workload,  discipline 
in  communication,  and  number  of  emergency  conditions  handled  successfully 
might  be  such  measures.  The  idea  of  product  measures  in  team  performance 
is  not  a  common  one  and  it  is  not  yet  adequately  defined.  There  fore,  it  is 
difficult  to  formulate  examples. 

This  approach  provides  a  profile  of  scores  on  several  attributes  or 
dimensions  to  characterize  a  team.  A  single  index  of  team  competence, 
capability,  or  readiness  is  desirable.  However,  the  usefulness  of  the 
index  is  a  function  of  the  extent  to  which  all  things  assigned  the  same  number 
behave  in  the  same  manner.  When  values  on  two  or  more  dimensions  are 
combined  to  give  a  single  index  of  status,  the  same  status  score  can  be 
produced  by  many  combinations  of  values.  Further,  the  weights  and 
combinatorial  rules  for  deriving  a  single  score  from  these  dimensions 
are  not  presently  known. 

Predictive  and  Diagnostic  Measures 

Measures  of  team  performance  must  be  related  to  measures  of  system 
performance;  the  ultimate  motivation  to  measure  team  performance  is  to 
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know  its  effect  on  system  performance.  There  are  two  kinds  of  team 
measures  of  interest:  predictive  measures  of  system  performance  and 
diagnostic  measures  of  team  operation. 

Predictive  measures  account  for  some  portion  of  the  variance  in  measures 

of  system  performance.  These  measures  may  have  to  be  identified 

on  the  basis  of  expert  judgments  of  criticalness,  importance,  or  impact 

on  the  system's  mission  objectives.  They  are  quantifiable  in  terms  of 

some  contingency  measure  such  as  correlation  or  linear  regression. 

Unfortunately,  many  uncontrollable  factors  affect  system  effectiveness 

and  limit  the  proportion  of  variance  that  any  single  factor  can  contribute. 

2 

A  C  team  can  perform  very  well  while  the  system  loses  the  battle  as  a  result 

2 

of  the  effect  of  other  factors.  Competence  of  a  C  team  is  a  necessary  but 
not  sufficient  condition  for  system  success. 

Another  way  of  evaluating  the  effect  of  team  competence  on  system  performance 
is  sensitivity  analysis  as  the  term  is  used  in  systems  analysis.  The  approach 
of  sensitivity  analysis  is  to  vary  values  of  an  input  variable  and  to  determine 
the  value  of  an  output  variable.  If  the  output  is  relatively  little- affected 
by  a  broad  range  of  variation  in  the  input,  then  the  system  is  insensitive 
to  the  input  variable.  If  there  is  a  wide  range  of  levels  of  team  competence, 
for  example,  which  have  little  effect  on  system  output,  this  result  indicates 
a  range  of  team  performance  that  is  acceptable  and  sets  a  lower  limit  on 
competence  that  must  be  attained  through  training.  Teams  better  than  that 
level  do  not  increase  the  level  of  system  performance.  Other  factors  affecting 
the  system  account  for  the  variance  in  system  performance  and  should  be 
addressed  first.  However,  if  there  is  a  high  positive  correlation  between 
team  and  system  performance,  then  more  team  competence  is  always  better 
and  selecting  the  level  to  be  attained  is  a  matter  of  cost-benefit  tradeoff. 
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Measures  which  are  diagnostic  or  indicative  of  difficulties  in  team 
operation  are  needed  to  improve  team  performance.  They  can  be 
interpreted  further  to  identify  the  kinds  of  operational  tasks  and  the 
conditions  under  which  team  performance  will  be  inadequate.  Ideal 
diagnostic  measures  will  identify  areas  of  weakness  or  unsatisfactory 
performance  and  the  deficiencies  in  knowledge  and  skill  which  underlie 
the  weakness:  they  will  identify  both  the  terminal  objectives,  on  which 
performance  is  unacceptable;  and  the  component  enabling  objectives, 
on  which  mastery  is  incomplete.  The  cost  of  resolving  power  to  the 
degree  of  specifying  knowledge  and  skill  may  increase  disproportionately 
with  level  of  detail.  Therefore,  it  may  be  necessary  to  select  the  level 
of  resolution  in  accordance  with  the  criticality  of  the  tasks. 

Determining  the  relationships  between  team  and  system  performance 
will  require  conceptualizing  the  information  flow  network,  of  which  the 
system  and  team  are  a  part,  and  interpreting  performance  in  relation 
to  the  functional  responsibilities  in  the  network.  The  amount,  relevance, 
and  timeliness  of  information  throughput  would  be  appropriate  measures 
of  the  network's  performance.  This  approach  lends  itself  to  analysis  by 
approaches  such  as  queuing  models,  linear  programming,  and  network 
analysis. 

Individual  capability  and  performance  are  also  a  factor  in  team  performance 
and  a  useful  system  of  performance  measurement  must  provide  these 
kinds  of  relationships. 
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Functional  relationships  should  be  developed  between  measures  of  team 
performance  and  measures  of: 

•  Combat  readiness  of  individuals 

•  Individual  competence  on  terminal  objectives  for 
operational  tasks  and  team  behaviors 

•  Individual  mastery  of  enabling  objectives  of  skills  and  knowledge 

Combat  readiness  is  a  complex  and  elusive  topic;  we  have  no  ready  or  easy 
answers.  A  starting  point  is  to  identify  necessary  behaviors  and  assume 
an  individual  to  be  combat-ready  if  he  demonstrates  mastery  of  them. 

A  difficulty  is  that  the  behaviors  should  be  assessed  under  something  like 
combat  stress  or  conditions  predictive  of  performance  under  combat  stress. 

The  development  of  performance  measures  should  start  during  system 
development.  The  body  of  data  about  teams  in  the  system  needed  during 
system  development  should  be  identified  in  order  to  produce  an  adequate 
battery  of  performance  measures  for  evaluation  of  the  learning  of  skills 
and  maintenance  of  proficiencies.  Analytical  techniques  should  also  be 
identified  for  use  in  analyzing  requirements  for  measurement  of  team 
proficiency  to  define  the  content  and  structure  of  performance  measures. 

Research  Strategy 

The  objective  of  this  area  of  research  is  to  develop  a  performance  measurement 
2 

system  for  TAC  C  teams.  Performance  measures  will  be  developed  which 
can  be  interpreted  in  terms  of  team  competence,  effect  on  system  effectiveness, 
and  diagnosis  of  individual  and  team  needs  for  remedial  training. 


73 


The  approach  consists  of  four  major  steps,  as  follows: 

1.  Identify  dimensions  of  performance  covering  both  operational 
tasks  and  team  behaviors. 

2.  Construct  product  and  process  measures  and  indexes  of 
team  performance. 

3.  Determining  the  relationships  between  team  measures  and 
system  effectiveness. 

4.  Develop  diagnostic  measures  of  team  performance  in  terms 
of  individual  deficiencies  and  needed  remedial  training. 

Operational  experts  will  have  to  be  used  to  identify  performance  dimensions 

and  evaluate  performance.  Candidate  measures  will  be  tried  out  by 

2 

application  to  operational  units  during  exercises.  Observation  of  C 
teams  will  also  be  necessary. 

A  simulation  facility  is  needed  so  that  teams  can  be  exercised,  observed, 
and  evaluated  in  selected  aspects  of  performance  under  realistic  scenario 
conditions.  The  facility  should  be  in-house  in  order  to  provide  continuity 
of  research,  adequate  control,  easier  access  to  operational  experts,  and 
availability  of  experienced  subjects.  The  facility  might  be  operated  under 
a  support  contract,  however. 

Specific  research  studies,  development  of  measuring  instruments,  and 
field  studies  for  data  collection  and  evaluation  can  be  done  on- contract. 
Contracts  will  provide  increased  manpower  and  specialized  skills  for  a 
short  term  such  as  one  or  two  years. 
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The  analytical  phase  can  be  done  either  on- contract  or  in-house. 

Availability  and  ease  of  access  to  the  experts,  and  sites  for  observation, 
should  be  the  determining  factor.  The  experimental  evaluation  should 
be  done  in-house  because  it  will  require  periods  of  extended  access  to  the 
simulation  facility. 

MAN-MACHINE  DESIGN  FOR  COMMAND  TEAMS 

The  human  factors  design  of  man- machine  interfaces  for  teams  in  general, 

2 

and  C  in  particular,  is  an  area  that  has  received  little  attention  prior 
to  the  last  few  years.  Conventional  human  factors  has  been  oriented 
toward  a  one- man /one- console  approach  in  designing  interfaces  and  it  is 
not  applicable  to  teams.  While  each  team  member  is  placed  at  a  workstation, 
the  person-person  interfaces  and  team  architecture  are  equally  important 
aspects  of  team  design.  Concurrent  design  for  all  positions  within  the 
team  as  a  unit  is  important  to  ensure  interoperability  among  positions. 

The  traditional  human  factors  approach  of  knobs  and  dials  for 

2  2 

operator  positions  is  another  limitation  when  applied  to  C  .  Much  of  C 
is  involved  in  processing,  evaluation,  and  interpretation  of  information. 
Information  exists  in  the  form  of  discrete  verbal  messages,  data  bases, 
tabular  formats,  and  graphic  situational  plots.  The  human  factors 
considerations  involve  knowing  how  the  user  uses  the  information,  what 
he  does  with  it,  and  how  his  use  can  be  supported  and  facilitated  effectively. 
These  topics  entail  consideration  of  mixed  initiative  dialogues  in  man- 
computer  interaction,  decision  support  or  aiding,  knowledge  representation, 
and  decision  processes. 
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The  approach  to  the  human  engineering  of  teams  must  rely  on  the  analysis 
of  information  flow  in  the  tactical  operations  of  the  team.  The  team  should 
be  viewed  as  a  distributed  network  for  information  processing.  The 
objective  in  team  design  is  to  partition  the  information  processing  of  the 
system  into  functional  areas  and  allocate  these  functions  to  team  members. 
The  functions  within  a  position  can  then  be  allocated  over  man,  machine, 
and  software.  The  design  of  the  team  must  also  include  the  interconnections 
among  positions  which  are  necessary  for  the  interdependence  and  coordination 
needed  to  support  the  operational  tasks. 

The  traditional  approach  of  designing  disjunctively  in  terms  of  man  or 
machine  is  no  longer  appropriate  and  the  preceding  paragraph  should  be 
modified  accordingly.  The  major  human  factors  problems  are  functions 
performed  jointly  by  man  and  machine.  Many  functions  are  performed 
by  computer-aided  man  or  man- aided  computer  rather  than  by  either 
component  alone.  Distributed  computing  systems,  intelligent  terminals, 
and  distributed  data  bases  with  multiple  access  are  accelerating  this  trend. 

There  is  a  multiplicity  of  potential  issues  related  to  the  architecture 

and  properties  of  teams.  Scanning  the  lists  of  architectural  dimensions 

and  functional  properties  in  Tables  3  and  4  will  readily  stimulate  thoughts 

of  some  of  them.  Selection  among  them  depends  on  an  analysis  of  some 
2 

specific  C  application  to  identify  a  type  of  team  of  relevance  to  Air  Force 
needs.  A  meaningful  research  question  exists  when  there  are  alternative 
designs  or  configurations  for  an  application. 
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Some  topics  concerned  with  the  nature  of  teams  have  been  selected  from 

recurrent  issues  in  the  literature  for  brief  discussion  of  potential  directions 

of  research.  They  are  summarized  under  team  composition,  interaction 

2 

between  task  type,  stability  of  team  composition,  and  automation  of  C 
functions. 

Team  Composition 


Team  composition  can  be  markedly  affected  by  the  nature  of  the  team's 
members  and  the  functions  performed.  The  existence  of  a  type  of  individual 
who  is  a  good  team  member  is  a  critical  question.  Comments  about  gcod 
and  poor  team  "players"  sire  commonplace  but  the  concept  is  wholistic 
and  has  little  predictiveness.  One  would  expect  to  find  the  differentiating 
attributes  of  good  team  members  in  the  team  skills  rather  than  the  operational 
tasks.  It  is  part  of  our  folklore  that  technical  excellence  is  not  sufficient 
for  good  team  functioning.  But  the  necessary  and  sufficient  attributes, 
skills,  and  knowledge  that  characterize  good  team  members  and  differentiate 
them  should  be  identified  and  evaluated  for  their  contribution  to  team 
effectiveness.  The  feasibility  of  selecting  or  training  for  these  characteristics 
can  then  be  investigated. 

Other  questions  of  team  composition  are  the  effect  of  heterogeneity  of 
functions  and  heterogeniety  of  competence  across  team  members  on  the 
output  and  processes  of  teams.  It  may  be  that  heterogeneity  forces  the 
development  of  good  team  skills  because  coordination  becomes  more 
critical. 
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Team  size  is  a  property  of  interest;  it  determines  cost  of  personnel 
and  support,  which  are  major  components  in  the  cost  of  ownership.  The 
changes  in  team  properties,  processes,  structure,  interaction,  and 
competence  with  changes  in  size  is  a  research  area  that  would  provide 
relevant  information.  Reduction  in  team  size  may  be  possible  as  developments 
in  solid-state  electronics  increase  the  amount  of  aiding  and  automated 
support  that  can  be  provided  in  "smart"  displays  and  terminals;  therefore, 
team  size  could  soon  be  a  key  issue. 

Interaction  of  Team  Type  and  Task  Type 

The  interaction  of  team  type  and  task  type  has  implications  for  the  design 
of  teams.  The  appropriate  type  of  team  may  be  different  for  procedural, 
semi-structured,  and  structured  tasks.  If  these  types  of  tasks  require 
different  team  processes  and  multiple  task  types  are  present,  it  is  necessary 
to  know  if  a  team  is  capable  of  some  kind  of  dynamic  reconfiguration  to 
adapt  to  these  conditions. 

It  is  also  desirable  to  know  in  emergent  situations  if  and  how  team  processes 
are  adaptable  to  changing  operational  conditions  and  events.  Adjustment 
to  variations  in  workload,  unexpected  even.s,  and  emergencies  require 
coordinated  changes  on  the  part  of  all  team  members.  Different  team  types 
may  be  differentially  effective  in  making  these  adjustments. 

Compensatory  mechanisms  in  team  processes  would  be  a  useful  property 
if  weaknesses  in  one  area  of  team  behavior  or  operational  tasks  can  be  offset 
or  compensated  for  by  strength  in  the  other  domain.  Similarly,  it  is  desirable 
to  know  if  team  members  can  coordinate  or  reassign  responsibilities  to 
compensate  for  weaker  members. 
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Stability  of  Team  Composition 


Stability  of  team  composition  is  undoubtedly  important  for  team  performance. 

Maturity  develops  under  stable  conditions  over  time.  The  team  that  stays 
together  develops  a  characteristic  kind  of  performance.  However, 
personnel  turnover  is  a  reality.  New  team  members  come  in  and  the  change 
is  assumed  to  be  disruptive.  Knowing  the  mechanism  and  effect  of  this 
change  on  team  performance,  it  may  be  possible  to  develop  ways  of 
introducing  new  members  that  minimize  the  disruptive  effects  of  personnel 
turnover.  Being  able  to  set  some  standards  of  individual  competence 
before  a  person  can  assume  a  position  in  the  team  is  a  potentially  significant 
way  to  reduce  disruption. 

The  practice  of  using  soft  vs  hard  teams  is  a  special  case  of  personnel 
turbulence.  Soft  teams  vary  in  membership  from  operation  to  operation,  \ 

depending  on  the  operational  conditions,  rules  for  forming  teams,  and 
availability  of  people  when  the  duty  roster  is  made  up.  Hard  teams  maintain 
the  same  roster  and  configuration  of  members  for  an  extended  period  of 
time.  The  effect  of  rotating  members  is  unknown  but  it  does  pose  some  man¬ 
agement  problems. 

For  example,  a  performance  score  for  a  soft  team  is  of  uncertain  utility 
since  the  particular  combination  of  individuals  is  unlikely  to  occur  again 
except  by  some  luck  of  the  draw  in  availability  of  manpower.  The  team 
score  cannot  be  generalized  to  another  team.  Further,  there  is  no  reason 
to  believe  that  a  team's  proficiency  can  be  derived  as  a  function  of  proficiency 
scores  assigned  to  individuals.  Finally*  if  meaningful  performance  measures 
cannot  be  assigned  to  teams,  it  is  impossible  to  develop  an  indication  of  the 
combat  readiness  of  a  unit. 
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It  is  also  possible  that  the  effect  of  deployment  of  soft  teams  elevates  the 
level  of  proficiency  that  can  be  achieved.  It  is  a  reasonable  assumption 
that  hard  teams  in  which  a  fixed  combination  of  individuals  exists  for  an 
extended  period  of  time  can  achieve  a  higher  level  of  integration  and 
coordination  and  these  levels  would  be  reflected  in  a  higher  team  proficiency. 
There  is  also  some  evidence  that  personnel  turnover  on  teams  is  accompanied 
by  degradation  in  performance.  Activities  that  involve  timing  would  be 
particularly  sensitive  to  this  factor. 

On  the  other  hand,  there  may  be  a  two-stage  process  at  work.  Hard  teams 
may  represent  an  early  stage  and  increasing  competence  arises  from  the 
increasing  strength  of  intra-team  connections.  However,  the  soft  con¬ 
figuration  may,  in  the  long  term,  result  in  still  higher  levels  of  proficiency. 
The  individuals  may  develop  very  generalizable  and  adaptable  team  skills 
as  a  result  of  experience  in  a  broad  range  of  team  combinations.  A  highly 
proficient  team  member  may  be  a  person  who  can  function  well  in  a  broad 
range  of  conditions. 

Research  could  be  initiated  to  determine  the  relationship  between  proficiency 

for  soft  and  hard  teams  and  between  individuals  and  team.  Such  a  project 

2 

would  be  long-term,  costly,  and  dependent  on  knowledge  of  C  and  team 
activities  that  we  do  not  have  at  this  time.  Such  a  program  could  provide 
data  on  the  advantages  and  disadvantages  of  the  two  team  compositions. 

Sound  personnel  policies  could  then  be  established. 
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Automation  of  C 


2 


Functions 


2 

The  trends  in  automation  will  have  a  significant  effect  on  the  design  of  C 
teams  for  future  systems.  The  effect  will  be  primarily  in  the  form  of 
decision  support  systems  incorporated  into  organizations  like  the  TACC 
and  CRC.  It  will  change  the  roles  and  functions  of  the  personnel  and  thus 
change  the  team  structure  as  well  as  the  operational  tasks. 

There  is  only  one  fully  operational  decision  aid  system  in  the  1979  Tactical 
Air  Force's  command  and  control  structure.  It  is  the  Tactical  Fusion 
Center  (TFC),  developed  under  an  ESD/MITRE  project  called  Operational 
Applications  for  Special  Intelligence  Systems  (OASIS).  It  provides  a  capability 
of  multisource  correlation,  information  aggregation,  and  display  of  theatre 
operations  and  intelligence  data.  It  aids  in  generating  a  "best  estimate 
of  the  situation"  for  the  Air  Component  Commander. 

There  will  be  an  increasing  number  of  computer-aided  support  systems  for 
2 

C  personnel  and  teams.  They  will  be  minicomputer  and  microprocessor 
systems  designed  for  ad  hoc  applications.  The  impact  and  value  of  these 
systems  on  manpower  and  training  requirements  are  unclear  and  uncertain 
at  this  time.  They  entail  major  significant  issues  of  man-computer  interaction 
on  which  there  is  not  yet  sufficient  data  to  provide  adequate  decision  for 
system  design  and  training.  They  will  also  force  changes  in  the  doctrine 
for  deploying  Air  Force  systems.  Introduction  of  a  computer  changes  the 
way  jobs  are  done  and  the  organizational  structure  rather  than  merely 
automating  activities  that  were  formerly  manual. 
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This  area  of  decision- aiding  is  a  potentially  fruitful  area  for  behavioral 

research.  However,  the  first  task  must  be  to  define  the  area  better.  The 
2 

C  team  model  will  provide  some  of  the  definition.  Other  subject  areas 

2 

include  understanding  the  decisions  and  analyses  that  C  personnel  make, 
the  information  used,  how  it  is  processed,  and  the  form  and  use  of  the  output. 
The  increased  automation  entails  extensive  man- computer  dialogues  which 
will  consist  of  mixed-initiative  interactions,  data  base  management,  natural- 
language  query  systems,  and  graphic /tabular  display  of  information  and 
situations.  The  applications  will  require  some  form  of  knowledge  representation 
and  the  parsing  of  dialogue  statements. 

The  design  of  a  decision  aid  is  a  complex  interdisciplinary  problem.  It 

2 

must  begin  with  selection  of  a  C  activity  for  which  aiding  would  be  beneficial 
and  which  is  aidable  with  some  available  techniques.  The  procedure  for 
developing  an  aid  is  well  summarized  by  Siegel  and  Madden  (Reference  15). 

A  diagram  of  the  procedure  is  presented  in  Figure  5.  They  have  also 
summarized  criteria  for  evaluating  the  utility  of  an  aid.  The  criteria  are 
presented  in  Table  6. 

The  complexity  of  this  area  in  methodology,  knowledge  of  required  decision- 
2 

aiding,  and  C  team  activities  indicates  that  research  in  this  area  should  be 
deferred  until  adequate  competence  in  these  elements  is  developed  and 
related  research  is  under  way.  Coordination  with  existing  programs  such 
as  those  at  ESD  should  be  done. 
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TABLE  6.  CRITERIA  FOR  EVALUATING  THE  UTILITY  OF  A 
DECISION  AID  (FROM  REFERENCE  15) 


Criterion 

Definition 

1. 

Internal  consistency 

Extent  to  which  the  constructs  of  the  aid 
are  marked  by  coherence  and  similarity  of 
treatment 

2. 

Indifference  to  trivial 
aggregation 

Potential  of  the  aid  to  avoid  major  changes 
in  output  when  input  groupings  or  conditions 
undergo  insignificant  fluctuations 

3. 

Correct  prediction  in  the 
extreme  (predictive  or 
empirical  validity) 

Extent  of  agreement  (correctness  of  pre¬ 
dictions)  between  the  aid  and  actual  perform¬ 
ance  at  very  high/low  values  of  conditions 

4. 

Correct  prediction  in  mid 
range  (predictive  or  empirical 
validity) 

Like  above  for  middle  range  values  of 
conditions 

5. 

Construct  validity 

Theoretic  adequacy  of  the  aid's  constructs 

6. 

Content  (variable  parameter) 
validity  (Fidelity) 

Extern  to  which  the  aid's  variables/ para¬ 
meters  match  real  life  conditions 

7. 

Realism  or  "face  validity" 

Extent  to  which  selected  content  matches 
each  attribute  included 

8. 

Richness  of  output 

Number  and  type  of  output  variables  and 
forms  of  presentation 

9. 

Ease  of  use 

Extent  to  which  an  analyst  can  readily  pre¬ 
pare  data  for,  apply,  and  extract  understand¬ 
able  results  from  the  aid 

10. 

Cost  of  development 

Value  of  effort  to  conceive,  develop,  test, 
document,  and  support 

11. 

T  ransportability-generality 

Extent  of  applicability  to  different  systems, 
missions,  and  configurations 

12. 

Cost  of  use 

Value  of  all  effort  involving  use  of  aid  in¬ 
cluding  data  gathering,  input,  data  pro¬ 
cessing,  and  analysis  of  results 

13. 

Internal  validity 

Extent  to  which  outputs  are  repeatable  when 
inputs  are  unchanged 

14. 

Event  or  time  series  validity 

Extent  to  which  aid  predicts  event  and  event 
patterns  ^ 
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The  objective  of  this  discussion  of  decision- aiding  is  intended  to  illustrate 
an  approach.  It  is  not  intended  to  be  a  comprehensive  or  even  illustrative 
discussion  of  current  research  and  knowledge.  That  discussion  would  be  a 
major  undertaking.  However,  the  approach  formulated  by  Siegel  and 
Madden  (Reference  15)  is  a  well-formulated  paradigm  for  design  and 
implementation  of  decision  aids. 


Research  Strates 


The  recommended  research  project  in  this  area  is  to  develop  a  set  of 

standardized  laboratory  tasks  and  exercises  which  can  be  used  in  laboratory 

research  studies.  A  capability  should  be  provided  so  that  experiments 

can  be  conducted  as  needed  on  team  operations,  team  processes,  and  the 

effects  of  specific  variables.  The  standardized  tasks  are  likely  to  be 

part- tasks  rather  than  a  global  simulation  of  the  operational  context  and 
2 

activities  of  C  teams.  Development  of  the  exercises  should  include  the 
hardware  and  software  on  which  to  run  the  studies  as  well  as  the  substantive 
content  of  the  tasks  and  experimental  design. 


The  objective  of  this  project  is  to  provide  realisitic  experimental  tasks 

that  can  be  used  flexibly  to  address  a  range  of  issues  and  hypotheses  about 

2 

teams,  manned  activities  in  C  ,  and  design  of  man- machine  interfaces 

2  2 
for  C  teams.  The  ability  to  generalize  to  operational  C  teams  and  systems 

is  an  important  criterion;  therefore,  representativeness  and  relevance 

of  the  standardized  tasks  are  important  attributes. 
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The  approach  consists  of  the  following  steps: 

2 

1.  Select  representative  functions  and  tasks  for  C  teams 

2.  Identify  team  variables  to  be  investigated 

2 

3.  Design  standard  C  team  tasks 

4.  Determine  the  specifications  for  hardware  and  software 
to  implement  the  tasks 

5.  Procure,  build,  install,  and  develop  the  hardware  and 
software 

6.  Test  and  evaluate  the  tasks  for  comparability  to  operational 
tasks 

The  purpose  of  this  project  is  to  develop  standardized  laboratory  tasks  which 
correspond  to  operational  tasks  of  teams.  These  tasks  and  their  software 
would  become  modules  in  a  simulation  library.  They  would  then  be  used  as 
components  in  assembling  scenarios  and  experimental  exercises. 

2 

This  project  would  interact  strongly  with  development  of  the  C  team  model. 
The  analyses  performed  to  develop  the  model  will  provide  data  on  tasks  and 
the  organization  of  tasks  on  which  this  project  can  build.  Similarly,  the 
standardization  of  tasks  will  feed  data  back  into  the  model  building. 

This  project  should  be  done  in-house  in  order  to  develop  and  sustain  an 
interest  in  and  a  sense  of  ownership  of  an  in-house  capability  and  to  ensure 
that  the  capability  is  responsive  to  the  user's  needs.  Some  specific  projects 
may  be  broken  out  for  contract  or  consultation  and  to  provide  support  in 
operation.  However,  the  main  work  and  integration  should  be  retained  in- 
house. 
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Decision-making  has  not  been  mentioned  as  a  research  topic.  Decision- 

2 

making  is  an  oft-cited  activity  in  C  .  However,  the  position  taken  here  is 

2 

that  the  role  of  decision-making  in  C  is  moot  at  best.  Far  fewer  activities 
are  decision-making  than  is  commonly  assumed  and  common  usage  of 
language  suggests. 

Command  and  control  consists  of  information  processing  in  a  wide  variety 
of  contexts.  These  activities  are  made  up  of  routines  and  algorithms  for 
sorting,  compiling,  estimating,  projecting,  allocating,  evaluating,  and  so  on. 
It  is  admitted  that  the  available  routines  are  incomplete,  often  too  slow,  and 
overwhelmed  by  tides  of  data  in  current  systems.  Human  judgment  is  used 
to  fill  the  gaps  and  simplify  the  tactical  problem  to  make  it  more  tractible. 
However,  the  remedy  lies  in  developing  more  and  better  routines  and  aids. 

2 

The  position  taken  here  is  that  attributing  a  C  activity  as  decision-making 
is  a  last  resource.  It  will  be  done  only  when  attempts  to  account  for  the 
activity  by  other  processes  have  failed. 

DEVELOPMENT  OF  TRAINING  REQUIREMENTS  AND  IMPLEMENTATION 
OF  TRAINING 

A  systematic  training  methodology  for  teams  (TMT)  needs  to  be  developed. 

The  methodology  will  be  broken  down  for  purposes  of  this  discussion  into 
the  areas  of  derivation  of  training  requirements,  instructional  strategy, 
sequencing  of  instruction,  and  performance  evaluation.  Specific  research 
topics  and  issues  will  be  discussed  in  each  area  except  performance  evaluation 
which  was  treated  in  a  prior  section.  Transfer  of  training  will  also  be 
discussed. 
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The  domain  of  TMT  must  encompass  subteam,  team,  and  superteam 
exercises.  A  complete  job  task  inventory  associated  with  a  position  must 
include  duties  and  responsibilities  at  all  levels  and  contexts  in  which  the 
individual  plies  his  occupational  speciality.  Therefore,  the  training 
requirements  must  be  analyzed  for  these  three  team  levels  as  well  as  the 
individuals'  behaviors. 

It  is  assumed  that  the  three  team  levels  represent  a  cumulative  aggregation 
of  tasks.  The  core  tasks  are  the  individual  ones  required  to  accomplish 
the  functions  and  responsibilities  of  a  position.  A  set  of  basic  team 
behaviors  is  added  at  the  subteam  level,  and  other  sets  are  added  for  team 
and  superteam.  The  addition  of  these  team  tasks  has  two  phases  of 
learning  the  task  behaviors  and  the  integration,  intermixing,  or  timesharing 
of  them  with  performance  of  the  operational  tasks. 

A  systematic  training  methodology  for  teams  does  not  exist.  Typical  team 
training  exercises  consist  of  operational  exercises  in  which  the  emphasis 
and  evaluation  are  on  operational  tasks.  Training  on  team  behaviors  is 
incidental  to  the  performance  of  the  operational  tasks,  and  the  extent  to 
which  it  occurs  is  highly  dependent  on  the  awareness  and  initiative  of  the 
instructor  in  perceiving  these  behaviors  and  providing  feedback  on  them. 

The  principal  factors  contributing  to  this  condition  are  the  lack  of  a 
taxonomy  and  performance  objectives  for  team  tasks  and  a  taxonomy  and 
training  objectives  for  team  behaviors.  These  factors  are,  in  turn,  a 
function  of  the  lack  of  a  conceptual  framework  for  team  behaviors.  Thus, 
the  behaviors  to  be  learned  and  the  types  of  learning  involved  cannot  be 
determined;  and  the  existing  literature  on  instructional  methods,  developed 
primarily  for  individual  performance,  cannot  be  used  or  evaluated. 


The  type  of  training  involved  is  referred  to  as  hands-on  or  practical 

experience.  The  character  of  team  training  has  a  strong  connotation  of 

performance  rather  than  cognition;  it  is  the  use  of  synergistic  behavioral 

interaction  to  facilitate  or  improve  the  productive  performance.  It  cannot 

be  taught  as  rules,  principles,  or  fixed  procedures;  it  must  instead  be 

understood  as  process,  flow,  and  interaction.  The  hands-on  training  takes 

2 

the  form  of  some  kinds  of  simulated  exercises.  These  exercises  in  C 
usually  consist  of  the  use  of  operational  equipment,  often  in-unit,  which  is 
stimulated  by  computer  in  response  to  a  computer- operated  driver  scenario. 
There  may  be  related  skills  and  knowledge  which  are  taught  in  the  verbal/ 
conceptual  format  in  classroom  settings;  the  use  of  the  simulated  exercise 
is  to  provide  a  vehicle  for  the  integration  of  these  skills  and  knowledge 
into  more  complex  tactical  behaviors.  However,  the  verbal/ conceptual 
component  is  minimal. 

Derivation  of  Training  Requirements 

There  is  a  need  for  methods  to  derive  training  requirements  from  the 
description  of  team  functions  and  tasks.  Existing  methods  as  represented  in 
ISD  are  inadequate  in  two  ways:  1 )  they  lack  the  behavioral  structure  to 
describe  team  tasks;  and  2)  they  have  not  been  adapted  to  the  semi- structured 
tasks  which  occur  in  emergent  situations. 

The  significance  of  the  semi- structured  character  is  that  the  tasks  or 
subtasks  fall  into  two  categories.  Some  are  fixed  procedures  or  algorithms 
which  are  standard  components  for  processing  information;  they  8 re  similar 
to  the  subroutines  in  a  computer  program.  The  other  tasks  are  variable 
in  the  sequence  and  nature  of  steps,  depending  on  the  kinds  of  driving  events 
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to  which  the  task  performer  must  respond.  However,  the  situation  is  not 
unorganized  or  chaotic.  The  performer  must  monitor  information,  recognize 
situations  requiring  action,  identify  the  appropriate  action  and  objective, 
select  and  combine  the  appropriate  behavioral  components,  and  adapt  the 
behavioral  sequence  to  the  characteristics  of  the  situation. 

Procedures  for  training  analysis  must  capture  the  data  to  describe  these 

elements.  The  lack  of  terminology  to  describe  these  tasks  will  be  remedied 

by  development  of  the  team  model.  The  team  behaviors  will  have  to  be 

2 

defined  in  the  context  of  C  activities  and  tasks  since  these  two  categories 

are  confounded  in  observing  the  behavior  of  command  teams.  This 

2  2 

interdependence  will  be  achieved  if  the  C  T  terminology  is  developed  in 

2 

specific  system  application  of  C  . 

Techniques  should  be  available  to  analyze  team  tasks  into  performance 
and  training  objectives  which  are  then  decomposed  into  skills  and  knowledge 
that  are  necessary  and  sufficient  to  produce  the  required  behavior.  The 
training  analysis  should  include  information  by  which  tasks  can  be  selected 
and  priorities  for  the  training  assigned  to  them.  This  information  should 
include,  at  a  minimum,  the  trilogy  of  task  characteristics:  criticality, 
difficulty,  and  frequency  of  performance. 

An  area  of  weakness  of  ISD,  at  least  as  it  is  typically  applied,  is  the  lack 
of  an  adequate  indicator  of  correspondence  between  the  derived  training 
requirements  and  job  performance  requirements.  Perceived  validity  of 
the  training  requirements  is  based  on  the  fact  that  they  start  from  an 
inventory  of  job  tasks. 
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However,  existing  task  analysis  is  a  linear,  deductive,  reductive  process. 
The  behaviors  of  job  tasks  are  broken  down  into  finer  levels  of  processes, 
knowledge,  and  skill.  The  resulting  training  requirements  are  then 
classified  by  the  type  of  learning,  organized  into  instructional  units,  and 
paired  with  instructional  strategies.  The  intermediate  and  final  products 
of  the  training  analysis  are  phenotypically  quite  different  from  the  job  task 
inventory,  and  it  does  not  seem  feasible  to  derive  a  practical  technique  for 
mapping  instructional  units  into  job  tasks.  One  could  reverse  the  analysis 
and  trace  the  skills  and  knowledge  back  to  the  job  tasks.  This  procedure 
would  be  unsatisfactory,  however;  errors  in  the  initial  decomposition 
would  validate  themselves. 

The  bridge  between  the  training  program  and  job  performance  requirements 
may  be  a  set  of  calibrated  or  graded  exercises.  The  rationale  underlying 
conventional  training  programs  presumes  a  building  block  approach  in  which 
components  of  knowledge  and  skill  are  learned  and  then  integrated  to 
provide  behavioral  capabilities  needed  in  the  job.  The  course  of  learning 
consists  of  a  progression  through  a  series  of  intermediate  behavioral 
objectives  that  are  increasing  approximations  to  terminal  performance 
objectives.  Achievement  of  these  objectives  is  evaluated  through  the  use  of 
performance  tests  or  simulated  exercises,  depending  on  the  complexity 
of  the  objective.  Each  test  and  exercise  should  be  described  at  a  minimum 
in  terms  of  a  performance  objective  and  standard,  the  knowledge  and  skill 
being  tested,  and  prerequisite  knowledge  and  skill. 
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However,  the  sequence  of  exercises  is  not  presented  as  or  demonstrated 
to  be  a  systematic  progression  toward  job  performance.  The  exercises 
are  treated  as  discrete,  nominal  entities.  The  sequence  of  exercises  is 
determined  by  the  sequence  of  instructional  units. 

2  2 

It  is  proposed  that  C  T  exercises  and  operational  activities  be 
calibrated,  scaled,  and  organized  in  a  multidimensional  problem  space. 

The  dimensions  of  this  space  are  continua  of  knowledge  and  skill. 

The  approach  presumes  that  items  of  knowledge  and  skill  can  be  ordered 
along  a  dimension.  Items  to  the  right-hand  side  represent  higher  levels  of 
competence  and  greater  difficulty.  A  primary  factor  in  sequence  is 
prerequisiteness:  prior  mastery  of  any  item  is  prerequisite  to  learning  the 
item  to  its  right  or  farther  out  on  the  dimension. 

A  hypothetical  example  of  a  continuum  might  be  "intercept  direction.  " 

The  difficulty  of  directing  intercepts  might  be  a  function  of  variables  such 
as  ratio  of  interceptors  to  hostiles,  number  of  hostiles,  number  of  inter¬ 
cepts/hour,  number  of  concurrent  intercepts,  and  type  of  aircraft.  A 
l/l  intercept  is  assumed  to  be  prerequisite  of  a  2/1  intercept;  five 
intercepts /hour  is  prerequisite  to  10  intercepts/hour,  and  so  on. 

Similarly,  interference  conditions  might  be  a  dimension  on  which  the  levels 
consist  of  degradation  of  communication  and  data  through  signal-to-noise 
ratios,  various  types  and  levels  of  ECM,  and  equipment  failures. 

Exercises  used  in  training  and  job  performance  tests  would  be  points  in 
this  multidimensional  space.  Positions  in  the  space  would  be  determined 
by  the  skills  and  knowledge  required  by  the  exercise.  The  highest  level  item 
required  within  a  dimension  would  determine  the  position  of  the  exercise. 
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The  relative  position  of  problems  in  the  space  reflects  the  sequential 
relationship  and  the  relative  difficulty  of  the  problems.  Level  of  competence 
is  a  monotonic,  increasing  function  of  distance  from  the  origin  perhaps 
calculated  as  a  vector.  The  last  exercise  in  a  sequence  should  be  a  job 
task  or  component  of  a  job  task.  Equivalent  exercises  for  any  point  can 
be  constructed  by  changing  the  context  of  the  exercise  or  attributes  on 
dimensions  that  do  not  interact  -with  the  dimensions  that  define  the  problem. 

2  2 

The  feasibility  of  constructing  a  C  T  problem  space  should  be  explored  by 
attempting  to  construct  one  for  a  limited  set  of  problems  that  involve  three 
or  four  dimensions.  The  approach  consists  of: 

2  2 

1.  Decompose  C  T  exercises  into  knowledge  and  skill. 

2.  Arrange  the  knowledge  and  skill  requirements  with  a  Gagne-type 
diagram. 

3.  Form  the  items  of  knowledge  and  skill  into  dimensions  and  scale 
the  items  in  order  of  competence,  difficulty,  or  prerequisiteness. 

4.  Place  the  exercises  in  the  space  and  determine  sequences  of 
exercises  that  constitute  pathways  to  a  terminal  job  performance 
objective. 

5.  Test  the  sequential  relationships  by  experimental  studies  using 
simulated  exercises. 

There  are  two  phases  of  the  project:  constructing  the  problem  space  and 
conducting  experimental  evaluation.  The  construction  phase  will  not 
require  special  physical  facilities.  It  is  basically  analytical.  Access  to 
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experts  and  observation  of  operational  and  training  exercises  will  be  needed. 
The  experimental  phase,  however,  will  require  a  simulator  and  appropriate 
scenarios.  The  specific  capabilities  of  the  simulator  cannot  be  defined 
until  more  is  known  about  the  exercises  to  be  used. 

Effective  instructional  pathways  through  the  problem  space  should  be 
developed  and  validated.  Pathways  are  sequences  of  problems  with  the 
use  of  specific  instruction  strategies.  Each  path  has  a  transition  time  to 
job  competence.  Skill  levels  are  defined  as  the  number  of  dimensions  on 
which  one  must  be  qualified  and  the  level  of  competence  that  must  be 
obtained.  Training  costs  and  efficiency  can  be  defined  in  terms  of  the  time 
and  resources  needed  to  reach  those  standards. 

This  idea  and  the  associated  research  might  get  so  complex  that  it  is 
difficult  to  manage.  It  would  also  heavily  rely  on  operational  experts  to 
provide  tne  information  needed  to  construct  the  dimensions.  The  existence 
of  constructability  of  the  dimensions  is  critical  and  uncertain.  Identification 
of  the  knowledge  and  skills  required  by  an  exercise  and  prerequisiteness 
can  be  done  using  Gagne's  (Reference  16)  methods  for  representing  the 
structure  of  a  subject  matter.  The  structure  of  the  subject  matter  is 
represented  as  a  hierarchical  network.  The  networks  consist  of  the  set  of 
knowledge  and  skills  necessary  to  execute  a  training  objective  and  the 
order  in  which  they  must  be  learned. 

The  merit  of  this  idea  is  that  it  introduces  order  into  a  subject  matter  and 
relates  performance  in  training  to  performance  on  job  tasks.  The  capability 
of  a  graduate  can  be  stated  in  terms  of  the  job  tasks  he  can  do  rather  than 
the  number  of  units  or  hours  of  instruction  completed. 
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Instructional  Strategy 


Given  training  objectives  that  contain  skills  and  knowledge  that  can  or  should 
be  learned,  they  must  then  be  paired  with  appropriate  instructional  methods 
to  provide  training.  The  combination  of  methods  and  the  procedure  for  a 
meaningful  unit  or  "chunk"  of  instruction  will  be  referred  to  as  an 
instructional  strategy. 

An  initial  step  in  selecting  an  instructional  strategy  is  classifying  the  skills 
and  knowledge  into  the  type  of  learning  involved.  An  appropriate  instructional 
method  can  then  be  selected.  It  is  commonly  assumed  that  different 
instructional  methods  or  strategies  are  differentially  effective  in  teaching 
specific  types  of  skills,  knowledge,  and  performance  objectives.  Further, 
it  is  assumed  that  there  is  a  best,  more  effective,  optimal,  or  preferred 
method  for  each  application. 

Those  relationships  are  known  to  some  extent  for  teaching  verbal,  conceptual, 
and  psychomotor  activities;  it  is  relatively  straightforward  to  identify 
appropriate  methods.  This  work  has  been  summarized  in  several  sources, 
such  as  Brock  (Reference  17),  Merrill  and  Tennyson  (Reference  18),  and 
Wulfeck,  Ellis,  Richards,  Wood  and  Merrill  (Reference  19). 

2 

These  relationships  are  applicable  to  C  team  training  but  are  currently 

indeterminate  since  the  skills,  knowledge,  and  performance  objectives  for 
2 

C  team  training  have  not  been  adequately  explicated.  Some  improvement 
can  be  made  by  extending  the  state  of  current  instructional  technology.  If 
C  team  activities  can  be  decomposed  into  a  taxonomy  of  knowledge  and 
skill,  they  can  then  be  referenced  to  appropriate  instructional  methods. 
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It  is  recommended  that  a  research  project  be  initiated  to  assess  the  use¬ 
fulness  of  existing  taxonomies  for  learning  types  and  instructional  strategies 
2  2 

for  C  T  .  The  approach  should  consist  of  selecting  one  or  more  of  the 
existing  taxonomies  and  classifying  performance  objectives,  skills,  and 
knowledge  for  team  behaviors  into  these  categories.  A  prior  or  concurrent 
effort  to  develop  a  taxonomy  of  team  behaviors  is  necessary.  The  resulting 
distribution  of  team  behaviors  into  learning  types  should  be  evaluated  for 
comprehensiveness,  ambiguity,  and  easy  use  of  the  process  and  rules  for 
classification. 

No  special  facilities  are  needed,  but  availability  of  operational  experts 

2  2 

and  the  opportunity  to  observe  C  T  operations  would  be  useful.  The 
research  is  appropriate  for  contract  but  could  be  done  equally  well  in-house 
if  the  appropriate  competencies  are  available. 

Sequencing  of  Instruction 

The  items  of  knowledge  and  skills  to  be  learned  must  be  grouped  into  units 
and  the  units  into  sequences.  The  items  are  clustered  into  homogeneous 
segments,  segments  into  lesson  plans,  and  lesson  plans  into  parts  of 
courses  or  courses. 

There  are  a  variety  of  approaches  to  organizing  and  sequencing  instructional 
materials  but  no  technique  that  is  adequate  for  practical  use  (Reference  20). 
Techniques  are  either  complex  and  theoretical  or  common  sense.  The 
common  rules  for  clustering  and  sequencing  are  at  the  level  of  relatedness 
of  content,  simple  to  complex,  general  to  detailed,  and  logical  sequencing. 
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2  2  2  2 
The  application  to  C  T  requires  sequencing  within  both  the  C  and  T  areas 

since  the  two  categories  are  interdependent.  It  is  probably  expedient  to 

2 

teach  team  behaviors  in  the  context  of  C  activities,  especially  for  initial 
training.  However,  it  would  be  heuristically  worthwhile  to  compare  the 
two  approaches:  teaching  team  behaviors  in  the  job  context,  and  teaching 
them  separately  in  a  neutral  context  free  of  job  content. 

It  is  possible  to  teach  team  behaviors  separately  in  a  game  context.  The 
use  of  some  job-free  medium  might  be  useful  to  highlight  the  interpersonal 
and  process  cues  and  to  attend  to  response  options  available  and  the 
behavioral  adaptions  required  to  adjust  to  conditions  within  the  team. 

If  team  behaviors  can  be  separated  from  the  job  content,  they  may  be  more 
generic  and  transferable  than  the  skills  of  operational  tasks.  The  techniques 
of  interaction  and  adjustment  are  the  same  in  a  wide  range  of  situations. 

We  postulated  that  team  behaviors  were  seperable  in  analysis  from  behaviors 
for  operational  tasks.  If  they  can  be  formulated  as  independent  of  operational 
content  or  cast  in  a  neutral  context,  they  can  then  be  investigated  and  taught 
separately.  The  potential  merit  of  this  approach  is  uncertain  and  a  research 
issue;  there  is  no  clear  a  priori  reason  to  expect  that  it  would  or  would  not 
be  advantageous. 

Clustering  and  sequencing  of  instructional  material  is  ultimately  dependent 
on  understanding  the  knowledge  structure  of  the  subject  matter.  Clustering 
and  sequencing  implicitly  impose  a  structure  on  the  training,  and  it  should 
be  compatible  or  noninterfering  with  the  knowledge  structure.  The 
knowledge  structure  does  not  exist  in  any  absolute  sense,  but  our  best 
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guess  is  that  it  is  determined  by  the  organization  in  the  job  system  and 
operational  environment.  Thus,  sequencing  is  part  of  the  correspondence 
between  training  and  job  requirements. 

2 

A  research  project  into  the  knowledge  structure  of  C  team  activities 
should  be  undertaken.  The  approach  should  include  review  of  current 
practices  and  approaches  to  sequence;  analysis  of  the  skills,  knowledge, 
and  behavioral  objectives  to  identify  relatedness  and  connectedness  among 
items;  transfer  of  training  between  learning  objectives;  and  identification 
of  sequences  of  learning  experiences  to  approach  terminal  performance 
objectives  comparable  to  job  performance  requirements.  These  require¬ 
ments  should  span  the  range  over  individual,  subteam,  team,  and  super¬ 
team  and  allocate  the  training  for  specific  learning  and  performance 
objectives  to  these  categories.  The  approach  is  very  similar  to  the 
building  of  the  problem  space  discussed  earlier  and  should  be  treated  as 
an  extension  or  application  of  it. 

Transfer  of  Training 

Transfer  of  training  is  a  basic  tool  in  achieving  effective  and  efficient 
training.  The  requirements  for  transfer  determine  the  correspondence 
between  the  conditions  of  the  job  and  the  training  situation  in  terms  of 
what  aspects  of  the  job  must  be  represented  and  the  degree  of  relatednoss. 
Transfer  can  also  be  a  factor  in  sequencing  of  instruction;  it  is  generally 
presumed  that  items  of  instruction  which  are  alike  in  content  to  a 
preceding  item  are  more  readily  learned  than  those  which  are  disparate. 
The  effective  use  of  transfer  also  requires  an  understanding  of  perceptual- 
motor  and  cognitive- motor  processes  of  the  learner. 
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Our  concepts  of  transfer  of  training  are  inadequate,  however,  for  any 
application  beyond  very  simple  or  contrived  applications.  Since  formula¬ 
tion  by  Thorndike  early  in  the  century,  they  underwent  considerable 
analysis  and  investigation  in  the  area  of  verbal  learning  but  with  little 
practical  accomplishment.  The  conditions  for  transfer  were  elaborated 
in  terms  of  common  elements  in  stimuli  and  responses  between  the  learning 
and  transfer  situations.  An  increasing  number  of  common  elements  were 
associated  with  increasing  similarity,  which  was  in  turn  associated  with 
increasing  transfer.  However,  both  common  elements  and  similarity 
have  been  resistant  to  operational  definition  and  measurement. 

l  An  extension  into  cognitive  similarity  was  attempted  with  an  approach  in 

which  stimuli  sharing  a  common  response  were  treated  as  similar  through 
mediation  of  the  shared  response.  For  example,  the  words  "parka"  and 
"igloo"  might  be  similar  through  mediation  of  the  common  association 
"cold;"  igloo  and  banana  on  the  other  hand  would  share  very  little 
between  them.  Several  investigators  have  compiled  lexical  atlases  of 
associations  as  a  means  of  identifying  mediating  responses.  However, 
this  approach  has  yielded  little  of  practical  value. 

► 

Measures  of  transfer  have  been  developed  in  terms  of  reduced  time  or 
errors  to  learn  new  material  or  reduced  time  or  errors  on  initial  exposure 
to  new  material.  It  seems  to  have  been  implicitly  assumed  that  the  slope 
and  asymptote  of  the  learning  curve  were  unchanged,  and  only  the  initial 
point  for  learning  the  new  material  was  affected. 
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These  measures  capture  much  of  our  intuitive  sense  of  the  accomplishment 
of  transfer.  However,  they  reveal  little  of  the  effect  on  the  trainee's 
capacity  for  performance.  For  example,  it  would  be  useful  to  know  how 
many  other  training  or  job  tasks  a  trainee  can  execute,  wholly  or  in  part, 
after  he  has  mastered  a  given  task  and  the  nature  of  the  changes  in  task 
conditions  under  which  performance  is  degraded  or  invariant.  If  a  student 
has  learned  how  to  calculate  the  probability  of  drawing  the  ace  of  spades 
but  cannot  calculate  it  for  the  ace  of  diamonds,  he  shows  a  deficiency  in 
transfer  even  though  he  might  learn  the  second  task  in  fewer  trials.  In 
contrast,  the  student  who  can  then  calculate  the  probability  of  throwing  a 
six  on  a  die  shows  a  high  level  of  transfer. 

Knowledge  about  the  range  of  conditions  and  applications  under  which 
behavior  is  available  following  given  learning  conditions  tells  us  something 
important  about  an  individual's  competence  and  capacity  to  adjust  in 
emergent  situations.  It  is  implicit  in  our  belief  that  the  inadequacy 
of  "book  learning"  is  its  failure  to  transfer  to  practical  situations  and  the 
corollary  belief  that  hands-on  exercises  are  superior  and  necessary.  It 
also  involved  attempts  to  determine  the  limits  at  which  behavior  breaks 
down. 

Concepts  of  transfer  have  been  static  for  the  past  30  years  or  more  and  have 
been  provided  very  little  in  conceptual  framework  and  algorithms  for  the 
formulation  and  analysis  of  applications  in  either  research  or  engineering. 
There  has  also  been  a  growing  trend  to  challenge  fidelity  requirements  on 
the  basis  of  a  general  scepticism  that  reproductions  of  operational 
conditions  have  been  overinclusive.  However,  we  do  not  have  data, 
principles,  or  algorithms  that  permit  us  to  say  which  cues  are  necessary. 
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useful,  or  relevant  for  cost-effective  training.  The  instigation  to  reduce 
fidelity  is  driven  more  by  a  desire  to  reduce  costs  than  by  analysis  of  tasks, 
training  objectives,  instructional  strategy,  and  conditions  of  transfer. 

2  2 

Research  on  transfer  of  training  for  C  T  will  have  to  be  done  in  that  con¬ 
text  and  is  therefore  dependent  on  progress  in  developing  the  conceptual 
framework  to  be  used  to  identify  and  describe  team  behaviors.  Its 

initiation  should  be  delayed  several  years  until  progress  has  been  made  on 

2  2 

the  structure  of  the  subject  matter  in  C  T  . 

Transfer  of  training  is  a  key  issue  in  the  design  of  training  programs. 

All  training  involves  some  degree  of  artificiality.  The  use  of  classroom 
instruction  for  verbal/ cognitive  material  is  highly  dissimilar  from  a  job 
context,  but  it  is  an  economical  method  of  delivery.  Simulation  also  has 
artificial  elements  which  are  simultaneously  its  advantages  and  disadvantages. 
It  gives  the  instructor  control  over  task  conditions  and  permits  him  to 
simplify  and  adapt  training  to  the  level  of  the  student.  It  permits  the 
creation  of  conditions  where  danger  or  risk  preempt  exposure  of  the  student 
on-the-job.  In  addition,  it  permits  creation  of  conditions  that  occur  rarely 
such  as  actual  combat  against  a  real,  armed,  intelligent  adversary.  The 
US  Army  has  been  having  success  with  engagement  simulation  for  this 
purpose. 

Still,  the  fidelity  of  these  training  situations  is  less  than  complete. 

However  realistic  they  seem,  the  threat,  enemy,  and  danger  are  not  real; 
the  trainee's  vulnerability  is  safeguarded  by  the  instructor.  Some  stimuli, 
particularly  contextual  ones,  are  attenuated  or  absent.  There  are  no 
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algorithms  for  estimating  the  degree  of  fidelity  needed  and  the  amount  of 
transfer  that  will  be  obtained.  They  are  probably  interactive  with  the  type 
of  tasks. 

At  the  present  state  of  technology  it  is  desirable  to  have  as  high  a  degree 
of  fidelity  as  possible,  even  though  the  increments  of  transfer  decrease 
in  the  upper  ranges.  Cost  considerations  create  a  force  to  reduce  fidelity 
as  low  as  possible  since  costs  and  level  of  fidelity  are  positively  related. 

The  growth  of  cost  is  possibly  exponential  in  the  upper  ranges  of  fidelity. 

Therefore,  the  specification  of  fidelity  requirements  affects  both  sides  of  the 
cost-effectiveness  ratio  for  training.  We  cannot  afford  to  neglect  it. 

SIMULATION  FOR  C2T2 

2  2 

The  use  of  simulation  for  C  T  will  require  technological  development  in 

three  areas:  1)  methods  for  development  of  simulated  exercises,  2) 

improved  techniques  for  the  management  of  simulated  exercises  for  training 

2  2 

and  research,  and  3)  specific  issues  needed  in  C  T  simulation. 

There  are  many  impressive,  existing  simulations  which  were  designed  and 
built  during  the  past  20  years.  The  US  Navy's  Tactical  Advanced  Combat 
Direction  and  Electronic  Warfare  (TACDEW)  facility  is  one  of  these. 
Generally,  these  simulations  are  costly;  their  capability  is  more  limited 
than  desirable;  they  lack  adequate  support  functions  for  instructors/ 
controllers;  they  are  inflexible  and  difficult  to  modify;  and  they  were 
developed  on  an  ad  hoc,  one-of-a-kind  basis. 
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The  routine  use  of  simulation  for  training  requires  removing  these  limitations. 
Recent  advances  in  computing  machinery,  software  design,  modeling,  war 
gaming,  and  training  make  it  possible  to  make  significant  advances  in  the 
capability  and  use  of  simulation. 

This  discussion,  of  simulation  has  two  aspects.  One  is  its  use  as  a 
research  tool;  the  second  is  its  use  for  training  of  command  teams.  It 
has  become  increasingly  necessary  to  rely  on  simulation  for  training 
because  rising  costs  and  shortages  of  equipment  have  produced  a  shortage 
of  live  flying  events  and  activities  for  training.  Furthermore,  simulation 
is  the  only  way  in  which  trainees  can  be  exposed  to  tactical  conditions 
and  force  ratios  characteristic  of  combat  in  the  period  of  1980  to  2000. 

Some  of  the  issues  to  be  discussed  are  common  to  both  applications  of 
training  and  research.  For  example,  the  procedures  for  determining 
requirements  for  and  designing  simulated  exercises  are  basically  the  same 
for  both  applications.  The  purpose  of  the  exercise  is  a  training  objective 
in  one  case  and  a  research  objective  in  the  other. 

Current  practice  in  the  use  of  simulation  and  development  of  training 
does  not  provide  systematic  procedures  for  deriving  simulation  requirements 
from  training  objectives.  Such  procedures  are  possible,  but  the  available 
technology  from  other  disciplines  has  not  been  used.  Current  practice  in 
simulation  tends  to  reproduction  of  the  operational  environment  and  equip¬ 
ment  in  as  much  detail  as  possible.  If  all  capabilities  and  events  of  the 
situation  can  be  generated,  then  all  system -related  behaviors  can 
potentially  be  elicited.  Consequently,  minimal  task  and  training  analyses 
are  needed  because  the  potential  for  all  system- related  behaviors  is 
inherent  in  the  simulation. 
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The  approach  is  costly  and  inefficient.  Fidelity  of  the  simulation  exceeds 
what  is  needed  for  training.  It  becomes  excessively  costly  since  the  costs 
of  simulation  increase  exponentially  with  increasing  fidelity.  It  is  also 
inefficient  to  use  since  it  is  usually  not  possible  to  isolate  parts  of  the 
simulation  selectively  to  meet  the  limited  needs  of  a  given  training  objective. 

Large-scale,  war- gaming  simulation  for  individual  and  team  training  has 
become  necessary,  while  the  advances  in  computers  through  solid  state 
electronics  are  making  their  use  feasible  and  affordable.  They  are  providing 
larger  memories  and  revolutionizing  software.  However,  we  need  better 
techniques  for  the  design,  use,  and  management  of  simulated  exercises 
and  the  information  they  can  generate. 

The  use  of  simulation  for  behavior^lly  related  issues  in  man-machine 
operating  systems  has  had  its  largest  application  in  flight  simulation. 

There  have  been  applications  to  ships,  submarines,  nuclear  power  plants, 
and  other  complex  or  dangerous  situations.  However,  flight  simulators 
have  been  the  most  extensive  and  sophisticated  developmental  programs. 

The  aerodynamic  equations  for  the  aircraft  did  much  toward  determining 
or  fixing  the  driver  scenario,  operating  procedures,  and  constraints  on 
the  simulated  system.  It  was  feasible  to  build  a  device  that  looks  and 
"flies"  realistically;  consequently,  aircraft  simulators  have  been  used 
successfully  for  training  in  flying  and  navigation. 

Simulation  has  been  less  adequate,  however,  in  representing  air  combat 
against  simulated,  intelligent  adversaries.  The  operational  conditions 
are  more  complex  and  less  structured;  therefore,  more  analytical  effort 
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must  be  put  forth  to  determine  what  should  and  can  be  simulated.  Simulation 
for  team  training  is  at  an  analogous  level  of  development.  We  need  ways  of 
representing  and  managing  this  greater  complexity  in  order  to  increase  the 
cost-effectiveness  of  training^ 

The  Navy  has  built  some  large  facilities  for  training  in  anti-submarine 

warfare  (for  example,  14A2  Team  Trainer)  and  sonar  operation.  They 

2 

are  stimulator/ emulators  rather  than  simulators.  Some  C  test  bed 

simulators  have  recently  been  developed  at  the  Naval  Ocean  System  Center 

(for  example.  Advanced  Command  and  Control  Architectural  Testbed- - 

ACCAT).  They  are  computer  systems  designed  for  development  and  test  of 

software.  They  were  not  intended  for  man-in-the-loop  simulation  and  have 

limited  capability  for  it.  However,  Poock  at  the  Naval  Postgraduate 

2 

School  has  made  some  use  of  it  for  human  factors  studies  in  C  (Reference  21 ). 

Methods  for  Development  of  Simulated  Exercises 


Systematic  procedures  for  defining  training  objectives  and  simulation 
requirements  for  simulated  combat  exercises  should  be  developed.  They 
should  be  applicable  to  exercises  for  individual,  subteam,  team,  and 
superteam  training.  The  procedures  would  provide  techniques  to  extract 
data  from  operational  requirements  of  systems,  tactical  scenarios,  and 
mission  requirements,  establish  performance  objectives,  and  analyze 
the  performance  objectives  into: 

1.  Training  objectives 

2.  Instructional  strategy 
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3.  Simulation  objectives  {consisting  of  the  events  and  conditions 
necessary  to  elicit  and  provide  adequate  exercise  of  the  skills 
in  a  training  objective) 

4.  Performance  measures,  diagnostic  analysis,  and  feedback 
techniques 

Simulation  and  training  are  two  distinct  methodologies  or  disciplines  that 
must  be  integrated  in  order  to  provide  effective  use  of  simulation  for 
training.  The  procedure  in  designing  and  developing  a  simulation  is 
to  produce  a  symbolic  or  graphic  replica  of  the  operational  situation  which 
is  driven  by  a  computer  and  responds  in  a  realistic  manner  to  events  in  the 
simulated  world.  If  used  in  training,  it  may  also  accept  or  sense  input 
from  the  trainee. 

The  procedure  in  established  methods  of  developing  training  courses  is  to 
develop  measurable  behavioral  objectives.  The  methodology  of  training 
development  must  be  extended  to  incorporate  simulation  requirements. 
Training  objectives  must  be  translated  into  simulation  requirements  for 
exercise  of  each  training  objective. 

Similarly,  there  is  a  standard  methodology  for  developing  simulation. 

It  must  be  extended  to  incorporate  sets  of  exercises  nested  within  a  larger 
scenario.  The  simulation  must  be  capable  of  flexibility  in  the  rapid 
configuration  of  modules  into  exercises  that  meet  the  requirements  of 
specific  training  objectives.  The  simulation  software  should  be  treated 
as  a  library  of  modules  which  can  be  called  up  to  compile  the  exercises 
needed  for  each  training  objective.  The  procedures  for  compiling  simulation 
exercises  should  be  within  the  capability  of  a  casual  computer  user. 
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As  the  complexity  and  scope  of  simulation  and  its  use  in  training  increase, 

instructor  support  becomes  increasingly  important.  Definition  of  the 

instructor's  role  is  needed  to  identify  actions  he  takes  and  tasks  he  performs 

in  setting  up  and  initializing  exercises,  feedback,  real-time  control, 

modification  of  the  scenario,  collection  of  performance  data,  processing 

of  performance  data  to  identify  competencies  and  deficiencies  in  skills, 

and  conducting  critique  sessions.  The  instructor  functions  should  be 

differentiated  into  roles  of  instructor,  controller,  programmers,  and 

computer  system  operator.  Identification  of  instructor  functions  and  some 

support  can  be  done  within  the  current  state  of  technology.  However,  the 

functions  can  be  only  minimally  incorporated  into  existing  simulations. 

Therefore,  progress  in  instructor  support  is  largely  dependent  on  the 

2  2 

research-oriented  development  of  simulators  suitable  for  C  T  . 

The  driver  scenario  must  provide  a  representation  of  the  threat  and  tactics 
at  a  level  of  fidelity  needed  to  support  the  training  objectives.  These 
attributes  are  derived  from  analysis  of  mission  and  operational  requirements. 
The  information  needed  is  functional  for  the  purposes  of  training  and  descriptive 
of  what  the  operational  personnel  do  and  how  they  do  it.  Existing  methods 
of  anaiysis  for  this  application  are  rudimentary  at  best. 

There  is  also  a  need  for  techniques  of  planning  and  analysis  to  define 
fidelity  and  functional  requirements  of  simulation  and  simulators.  This 
issue  involves  knowing  what  and  how  to  simulate  and  translating  these 
requirements  into  features  of  scenarios,  fidelity  in  representation, 
instructional  strategies,  software  architecture,  and  functional  capabilities 
of  hardware.  This  information  must  be  generated  by  analysis  of  performance 
objectives.  It  is  integrated  into  functional  capabilities  of  a  simulation 


facility  which  can  in  turn  be  translated  into  design  concepts  and  engineering 
specifications  from  which  a  simulator  or  simulation  facility  can  be  built. 

Establishing  fidelity  requirements  begins  with  identifying  cues  and  behaviors 
which  are  necessarj'  or  beneficial  in  the  performance  and  learning  of 
operational  team  tasks.  This  information  should  be  integrated  into  the 
specifications  for  simulated  exercises.  These  requirements  should  specify 
the  attributes  of  the  operational  situation  which  must  be  representative 
or  functional  and  the  level  of  realism  needed  to  support  specific  training 
and  performance  objectives. 

The  simulation  requirements  should  be  interpreted,  decomposed,  and 
extrapolated  into  the  definition  of  the  model  bases,  data  bases,  and  algorithms 
necessary  to  support  the  exercise,  performance  measurement,  and 
training  analysis.  The  model  bases  will  include  representations  of  the 
weapons  and  platforms  employed  in  the  exercises,  terrain,  and  atmospheric 
conditions  and  the  events  of  behavior  and  system  operation  which  must  be 
sampled,  monitored,  and  controlled.  This  type  of  analysis  may  require 
the  integration  of  concepts,  data,  and  techniques  from  mission  analysis 
and  instructional  technology  to  identify  the  data  needed  and  the  level  of 
resolution  and  detail. 

Management  of  Simulated  Exercises 

There  are  also  problems  in  the  management  of  simulation  exercises  and 
facilities.  Functions,  duties,  and  tasks  performed  by  instructors, 
controllers,  players,  role  players,  and  support  personnel  must  be  defined. 
Further,  the  man-machine  interfaces  and  computer  support  needed  for 
each  individual  must  be  determined.  Techniques  are  required  to  extract 
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simulation  requirements  needed  to  support  the  instructional  strategy  for 
each  training  objective  or  family  of  training  objectives.  It  is  assumed 
that  the  simulation  personnel  will  be  given  the  training  needed  and  develop 
the  instructional  strategy  in  collaboration  with  the  training  user. 

Modification  and  update  of  the  simulation  facility  in  accordance  with  changes 
in  simulated  systems  or  feedback  from  evaluation  of  the  training  program 
will  be  necessary.  Analytical  aids  will  be  needed  to  assist  simulation 
personnel  in  determining  the  impact  of  the  changes  and  making  the  necessary 
modifications. 

Training,  briefing,  and  design  for  support  personnel  are  very  important 
factors  in  successful  operation  of  simulation.  It  is  especially  important 
for  support  personnel  who  role-play  components  in  the  scenario.  Simulated 
aircraft,  for  example,  are  manually  controlled  by  operators  at  consoles. 
Their  actions  generate  simulated  radar  returns  for  the  interceptor  aircraft . 
The  operators  are  typically  enlisted  personnel  who  are  not  flight  rates; 
they  have  little  knowledge  of  the  aerodynamics  of  aircraft  and  limitations 
such  as  maximum  turn  rates  and  tactics. 

Designing  for  these  personnel,  as  integral  to  the  simulation,  will  enhance 
the  effectiveness  of  the  simulator  for  training.  There  are  two  available 
approaches:  training  for  the  role  player  and  reallocating  some  functions 
in  the  simulation.  Software  can  be  designed,  for  example,  to  impose  limits 
on  the  movements  of  platforms  and  even  to  execute  some  maneuvers  on 
command.  Automation  of  aspects  of  aircraft  maneuverability  will  require 
aircraft  models  and  data  bases  for  simulating  platform  motions  and 
atmospheric  conditions  that  affect  flying.  This  level  of  automation  would 
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require  design  of  a  new  operator- simulator  interactive  interface.  It 
would  reduce  training  requirements  in  the  control  of  the  simulated  aircraft. 
However,  the  total  effect  of  a  redesign  is  uncertain  since  added  capability 
and  operator-computer  interaction  may  increase  the  scope  of  the  operator's 
responsibility. 

Research  studies  will  also  require  changes  in  standard  experimental 
practices.  Participants  as  subjects  would  have  to  be  sophisticated  in  the 
operational  activities  to  be  able  to  generalize  the  studies  to  operational 
problems.  Therefore,  a  core  of  long-term,  semi-professional  subjects 
would  be  required.  They  could  be  supplemented  by  military  subjects  on 
short-term  assignment.  The  "standard"  college  sophomore  might  be  used 
occasionally  to  test  hypotheses  about  basic  psychological  processes  if  the 
hypotheses  become  significant  or  critical  issues.  Extended  scenarios 
spanning  hours  and  days  of  operations  will  be  necessary  to  provide 
adequate,  realistic  baseline  data.  Multiple,  equivalent,  or  parallel 
exercises  and  scenarios  will  be  required. 

Better  methods  for  the  conceptualization,  management,  and  analysis  of 
studies  must  be  developed.  The  costliness  of  extended  periods  of  data 
collection  makes  it  necessary  to: 

•  Test  several  hypotheses  concurrently. 

•  Conceptualize  behavioral  processes  so  that  analysis  of  the  protocols 
of  behavior  and  system  events  will  provide  multiple  data  points 
that  are  equivalent  for  testing  hypothesis. 
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•  Provide  management  tools  for  the  study  director,  controllers, 

and  support  personnel  to  enhance  their  capacity  to  use  the  behavioral 
data  base  generated  in  a  study  to  provide  answers  to  the  questions 
and  hypotheses  for  the  study. 

•  Provide  management  tools  for  the  director  and  controllers  to 
keep  track  of  the  state  of  the  experiment  in  real  time  and  make 
decisions  in  accordance  with  the  objectives  of  the  study. 

2  2 

Specific  Needs  in  C  T  Simulation 

The  increasing  costs  of  live  exercises  have  generated  a  need  for  simulated 

training  approaching  the  realism  of  live  events.  However,  the  scarcity 

2  2 

of  suitable  equipment  and  the  low  priority  of  both  C  and  T  in  budgeting 
indicates  the  need  for  research  to  provide  data  on  comparative  benefits 
and  costs  of  live  and  simulated  events.  The  use  of  simulation  would  also 
entail  significant  expenditures  to  provide  enough  devices  of  adequate 
capability.  Further,  the  comparative  evaluation  would  be  expensive  in  the 
requirement  of  live  flying  which  is  not  now  a  part  of  the  operating  budget. 

The  studies  can  be  both  analytic  and  experimental.  Cost  data  on  live  events 
can  be  obtained  analytically.  The  idealized  approach  would  be  to  determine 
the  gains  in  proficiency  for  numbers  of  live  and  simulated  events  for  given 
tasks  or  families  of  tasks.  The  numbers  of  events  can  also  be  translated  into 
costs.  The  data  generated  would  permit  determination  of  the  comparative 
value  of  the  two  kinds  of  events  and  the  strengths  and  weaknesses  of  each 
method  for  given  tasks. 
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There  are  also  some  significant  deficiencies  in  current  simulation  in  the 
areas  of  interactive  ECM  and  sensor  management.  Realistic  simulation  of 
ECM  and  its  incorporation  into  man-in-the-loop  simulation  has  been  a 
chronically  unsatisfactory  situation.  The  shortcomings  have  been  in 
representation  of  the  output  characteristics  of  interference,  complexity 
of  concurrent  use  of  multiple  types  of  ECM,  realism  of  counter-countermeasures 
which  operators  can  employ,  and  the  software  intensiveness  of  the  simulations. 

Analysis  should  be  initiated  to  determine  the  requirements  for  adequate 
simulation  of  ECM,  performance  objectives  needed  to  provide  operators 
competent  in  operations  under  ECM  conditions,  and  training  objectives. 
Cost-benefit  analysis  can  then  be  performed  to  evaluate  alternative 
approaches. 

Development  of  training  objectives  for  sensor  management  also  entails 
establishing  performance  and  training  objectives  from  which  the  training 
requirements  can  be  established  and  evaluated.  Sensor  management 
involves  team  coordination  consisting  of  development  of  a  deployment  plan 
for  sensors  by  the  battle  staff,  its  implementation  by  the  console  and  sensor 
operators,  and  maintenance  of  system  quality. 

Performance  in  ECM  and  sensor  management  must  be  evaluated  by  an 
expert,  instructor,  or  controller  against  some  criterion  of  adequacy.  They 
cannot  be  evaluated  against  criteria  of  an  adversary's  response,  increased 
attrition,  or  changes  in  probability  of  kill,  P^)* 
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These  areas  must  be  provided  for  in  the  design  of  a  simulation  so  that 
adequate  provision  can  be  made  for  models,  algorithms,  performance 
measures,  and  interaction  with  the  driver  scenario. 

Research  Strategy 

Given  the  broad  scope  and  complexity  of  the  issues  that  must  be  addressed 
in  designing  a  simulation  facility,  it  is  recommended  that  a  study  be  under¬ 
taken  to  define  in  detail  the  needed  and  desired  functional  capabilities 
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for  simulation  facilities  devoted  to  C  T  .  The  objective  should  be  to  develop 
a  Type  A  System  Specification  in  accordance  with  MIL-STD-490  (Reference 
22).  The  study  should  define  the  uses  to  which  the  facility  will  be  put,  the 
functions  which  must  be  performed  to  satisfy  these  uses,  support  functions 
and  personnel  needed  to  operate  and  manage  the  facility,  and  a  system 
concept  and  configuration  of  the  system. 

The  approach  consists  of  the  following  major  steps: 

1.  Identify  the  types  of  training  exercises  to  be  conducted. 

Description  of  the  exercises  should  include  the  numbers  and  types 
2 

of  players,  C  functions  and  tasks  to  be  performed,  performance 
measures  and  other  data  collection,  functions  of  operational 
equipment  which  must  be  simulated,  and  the  tactical  operating 
procedures  involved. 

2.  Identify  the  experimental  issues  and  questions  to  be  addressed, 
the  types  of  experimental  designs  to  be  employed,  and  the  kinds 
of  data  analysis  to  be  performed.  Description  of  the  experiments 
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should  include  the  numbers  and  kinds  of  people  involved,  variables 
to  be  included,  functions  and  tasks  to  be  performed,  performance 
measures  and  other  data  collection,  and  the  types  of  operation 
equipment  which  must  be  simulated. 

3.  Develop  tactical  scenarios  representative  of  the  kind  to  be  used 
in  training  and  research  studies.  The  scenarios  should  include 
temporal  course  of  events,  permissible  tactics  and  maneuvers, 
types  and  numbers  of  platforms  involved,  parameters  and  functional 
characteristics  for  weapons  and  sensors,  tactical  constraints, 
geopolitical  conditions,  and  environmental  conditions. 

4.  Identify  support  personnel  who  will  operate  the  simulation  facility 
including  computer  operators,  maintenance  staff,  role  players, 
simulation  controllers,  and  director.  Define  role,  functions,  and 
tasks  for  each  type  of  individual.  Identify  man-machine  interfaces 
needed,  types  of  information  required,  and  control  actions  needed. 

5.  Identify  major  categories  of  software  needed  including  models  of 
systems  and  platforms,  supporting  data  bases  and  algorithms 
required  to  perform  computations  necessary  to  run  the  scenarios, 
exercises,  and  experiments.  These  software  requirements  should 
be  translated  into  a  preliminary  software  architecture. 

6.  Develop  a  preliminary  functional  network  for  the  simulation 
facility  including  a  system  concept,  configuration  of  major 
components,  required  supporting  utilities,  and  functions  to  be 
implemented.  This  statement  is  the  Type  A  System  Specification. 

It  is  the  design  objective  for  subsequent  development  leading  to 
detailed  hardware  and  software  specifications  from  which  procure¬ 
ment  can  be  done. 
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The  approach  was  used  for  development  of  the  Naval  War  Gaming  System 
at  the  Naval  Air  College,  Newport,  R.  I.  Relevant  documents  are  the 
Statement  of  Work  (Reference  23),  Representative  Scenarios  for  NWC 
War  Gaming  (Reference  24),  and  Program  Performance  Specification 
(Reference  25).  The  last  document  takes  the  design  beyond  the  Type  A 
level,  but  it  is  useful  in  illustrating  the  categories  and  kind  of  information 
needed. 

Similar  kinds  of  analyses  were  done  in  procuring  the  Combined  Arms 
Tactical  Training  Simulator  (CATTS)  which  has  been  in  operation  at 
Combined- Arms  Training  Development  Activity  (CATRADA),  Fort 
Leavenworth,  Kansas.  A  basic  document  for  that  system  is  the  Specification 
for  Trainer  (Reference  26). 

This  study  will  not  require  special  facilities,  but  it  will  require  an  appro¬ 
priate  mix  of  skills  among  the  personnel.  Systems  analysts,  software 
designers,  computer  engineers,  human  factors  engineers,  operational 
specialists  representing  the  user  community,  and  training  specialists 
should  all  be  represented. 

The  work  should  be  done  jointly  in-house  and  on- contract.  Some  steps, 
such  as  defining  the  types  of  training  exercises  and  experimental  studies, 
require  heavy  involvement  of  military  personnel  and  integration  of  their 
requirements.  Therefore,  it  should  be  an  in-house  effort  with  some  support 
from  consultants,  perhaps.  Other  steps  can  be  formulated  so  that  major 
parts  of  them  can  be  done  on- contract. 
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A  second  study  is  recommended  in  the  area  of  determining  simulation 
requirements  for  ECM  and  sensor  management.  The  approach  consists  of: 

1.  Determine  the  effects  of  ECM  and  sensor  management  on  job 
tasks.  The  tasks  affected  and  the  changes  in  operating  procedure 
should  be  identified. 

2.  Prepare  performance  and  training  objectives  for  operations 
under  ECM  conditions  and  sensor  management. 

3.  Identify  adequate  simulation  capability  and  fidelity  requirements 
to  support  training  objectives. 

4.  Identify  means  of  incorporating  the  simulation  for  ECM  and 
sensor  management  into  standard  exercises. 

5.  Conduct  cost-benefit  analysis  of  the  simulation. 

6.  Recommend  a  simulation  approach. 

This  work  can  be  done  either  on-contract  or  in-house  depending  on  the 
availability  of  competence  in  ECM  and  sensor  management.  No  specialized 
facilities  are  needed. 

PERSONNEL  REQUIREMENTS 

The  most  significant  research  issue  encountered  involving  personnel  is  the 
selection  of  weapons  controllers.  There  are  currently  no  entry-level 
aptitude  requirements.  Determining  and  setting  minimal  aptitude  require¬ 
ments  could  reduce  training  time  and  increase  the  ability  of  controllers. 


116 


At  this  time,  we  do  not  know  the  behavioral  processes  of  weapons  control 
well  enough  to  begin  to  identify  abilities  and  aptitudes  as  candidate  variables 
for  selection.  Expert  judgment  of  peer  nomination  would  be  necessary  even 
to  identify  good  and  poor  controllers  given  the  status  of  performance 
measurement  and  evaluation.  The  differentiating  factors  could  lie  in 
either  the  operational  tasks  or  the  team  behaviors. 

Setting  selection  criteria  presumes  that  there  are  attributes  of  ability  and 
life  history  that  are  related  to  success  or  competence  in  job  performance. 
Adequate  measures  of  job  performance  must  also  be  available  to  serve 
as  criteria  for  determining  the  validity  of  predictor  variables. 

This  issue  is  confounded  with  the  problem  of  the  availability  of  manpower 
with  given  qualifications  in  sufficient  number  to  satisfy  the  manning  require¬ 
ments  of  Air  Force  systems.  The  system  designer  or  manager  might 
have  to  choose  between  a  larger,  more  heterogeneous  supply  of  manpower 
with  lesser  competences  and  a  smaller,  more  homogeneous  supply  with 
greater  competence.  If  the  choice  is  equivalent  to  selecting  between 
overmanning  with  incompetents  and  undermanning  with  competents,  then 
the  choice  may  well  be  a  lose-lose  situation. 

Establishing  criteria  for  selection  of  the  weapons  controller  is  within  the 
current  state  of  psychometric  technology.  However,  it  is  a  time-consuming, 
labor-intensive  undertaking  which  cannot  be  completed  quickly.  The 
initial  step  to  be  taken  consists  of  assessing  the  difficulty  of  the  problem.  A 
survey  of  the  weapons  controller  job  and  personnel  should  be  undertaken 
to  identify  potentially  relevant  attributes  and  job  tasks,  skills,  and 
knowledge  which  might  differentiate  among  levels  of  performers.  The 
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existence  and  availability  of  suitable  measuring  instruments  for  predictor 
and  criteria  variables  and  the  need  for  additional  instruments  to  be 
constructed  should  be  determined.  Finally,  a  cost-benefit  analysis  for 
development  and  use  of  a  selection  battery  should  be  performed.  If  the 
effort  is  judged  worthwhile,  a  development  plan  would  then  be  instituted. 

Research  on  command  team  behaviors  would,  of  course,  yield  information 

2 

on  the  behavioral  processes  in  C  activities  which  can  contribute  to  iden¬ 
tifying  potential  selection  variables.  This  point  can  be  illustrated  by  a 

2 

possible  research  topic  of  spatial  perceptual  organization  in  C  behaviors. 

2 

A  critical  capability  of  weapons  controllers  and  perhaps  all  C  personnel 

is  the  ability  to  form  a  dynamic  spatial  construct  of  the  dispersion  of 

forces,  their  movement,  relationship  to  each  other,  and  the  observer's 
2 

orientation  to  the  C  graphic  plot.  This  organization  is  basic  to  giving 

direction  such  as  range  or  bearing  from  the  perspective  of  the  operator  of 

2 

a  platform  rather  than  the  C  operator.  This  construct  is  a  three-dimensional 

2 

geometric  one  extended  through  time.  The  C  operator  must  use  this 
spatial  structure  to  select  paths,  anticipate  conflicts,  and  extrapolate 
intercept  points. 

This  capability  was  identified,  during  a  discussion  with  a  respondent  to 
the  interviews,  as  one  that  differentiates  among  good  and  poor  controllers; 
its  acquisition  is  a  critical  point  in  training.  Observation  of  weapons 
directors  provided  additional  information  indicating  that  good  controllers 
can  operate  from  orientations  both  outside  and  inside  the  cockpit  of  an 
aircraft. 
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A  similar  capability  is  required  of  civilian  air  traffic  controllers.  There 
is  some  related  information  from  the  Ohio  State  simulation  project  on  air 
traffic  control.  For  example,  a  board  game  was  developed  as  a  training 
aid  for  teaching  this  skill.  There  are  similar  problems  of  spatial  orientation 
for  operators  of  remote  manipulators.  The  capability  might  be  a  basis  for 
selection  of  weapons  controllers. 

Skill  level  may  also  be  an  important  variable  affecting  the  use  of  simulation. 
This  relationship  is  suggested  by  the  findings  of  a  study  of  performance 
measurement  in  high- skill  specialties  (Reference  27).  Job  sample  performance 
tests  were  used  to  assess  performance  in  a  highly  skilled  electronics 
maintenance  specialty.  Valid  and  reliable  tests  were  constructed.  However, 
the  requirements  in  equipment,  facilities,  and  standardization  limits  its 
use  to  an  ideal  location  such  as  a  school. 

If  the  finding  is  applicable  to  operator  as  well  as  maintenance  jobs  and 
to  training  as  well  as  performance  evaluation,  then  the  study  has  serious 

2  2 

implications  for  C  T  .  Training  for  highly  skilled  personnel  takes  place 
predominantly  in  their  units  and  through  on-the-job  training  (OJT).  That 
training  is  not  sufficiently  controlled  to  be  beneficial.  Alternative 
approaches  may  be  prohibitively  costly  and  disruptive  of  unit  operations. 

The  recommended  personnel  research  is  a  feasibility  study  of  setting 
selection  criteria  for  weapons  directors.  The  approach  consists  of  surveying 
the  weapons  director  career  field  for  potential  attributes  and  the  criteria 
selection,  assessing  the  impact  on  the  career  field  in  terms  of  changes  in 
performance  and  availability  of  qualified  personnel,  and  determining  the 
costs  of  developing  and  implementing  a  selection  battery. 


Standard  data  processing  facilities  for  psychometric  research  are  needed. 
The  key  factor  is  the  availability  of  qualified  military  psychometrists. 
Therefore,  the  study  can  be  done  best  in-house. 

SUMMARY  OF  RESEARCH  RECOMMENDATIONS 

The  issues  and  problem  areas  identified  during  this  study  have  been 
discussed  and  16  specific  research  projects  have  been  recommended. 
These  projects  are  listed  in  the  right-hand  column  of  Table  7  and  are 
cross-referenced  by  parenthetical  numbers  to  the  Issue  and  Problem 
Areas  listed  in  the  left-hand  column. 

Each  project  has  been  described  in  terms  of  objective,  approach,  and 
resources  required.  These  descriptions  may  be  found  in  Chapter  IV, 
where  the  topic  emerges  in  the  discussion. 

The  projects  are  listed  below  with  a  brief  statement  of  the  objective  and 
intent  and  are  also  cross-referenced  by  parenthetical  numbers  to  the 
Issue  and  Problem  Areas  in  Table  7. 

2 

Field  Study  for  Further  Definition  of  Deficiencies  in  C  Team 
Performance  and  Training  (All  issues  and  problem  areas) 

The  objective  is  to  obtain  more  precise  definition  of  the  deficiencies 
found  during  this  study  by  using  more  objective  and  focused  evaluation  of 
each  issue.  Better  estimation  of  the  impact  on  effectiveness  and  cost- 
benefit  data  could  be  obtained. 
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TABLE  7.  RECOMMENDATIONS  FOR  LONG-TERM  RESEARCH 


2 

Construct  a  Model  for  C  Team  Performance  in  CRC, 

AWACS,  or  TACC  Type  of  Organization  (1 ) 

2 

The  objective  is  to  take  the  first  step  in  developing  C  team  performance 

models.  The  purpose  of  the  model  is  to  provide  a  conceptual  framework 

2 

and  terminology  for  describing  and  analyzing  the  behaviors  of  C  teams. 

The  approach  consists  of  observation  of  team  operations  and  analysis  of 

2 

the  technical  literature  on  teams  and  C  to  develop  and  test  a  limited 
model  based  on  a  selected  system. 

Develop  Procedures  for  System  and  Task  Analysis,  Compatible 
with  MIL-H-46855  and  ISP,  to  be  Used  in  Design  of  Man-Machine 
Components  for  C2  Teams  (3,  4) 

The  objective  is  to  develop  tools  for  generating  and  analyzing  behavioral 
data  to  design  man-machine  interfaces,  personnel  subsystems,  and  training 
during  system  development  and  maintenance  of  system  readiness.  These 
tools  will  be  adaptations  or  extensions  of  existing  methods  to  the  less 
structured  situations  of  command  and  control  systems  and  team  performance. 

2 

Develop  a  Performance  Measurement  System  for  C  Teams 
which  Assesses  Team  Competence  Effect  in  System  Effectiveness 
and  Needs  for  Remedial  Training  (2 ) 

The  objective  is  to  provide  more  objective  and  systematic  measure  of 
2 

performance  of  C  teams.  These  measures  are  intended  to  provide  an 
index  of  team  competence,  to  be  able  to  interpret  in  terms  of  impact  on 
system  effectiveness,  and  provide  diagnosis  of  team  and  individual 
training  needs. 
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2 

Compare  "Soft"  vs  "Hard"  C  Teams  in  Performance 
of  Representative  Cz  Tasks  (1,  2) 

Soft  teams  do  not  have  a  stable  membership,  but  team  members  change 
between  exercises.  The  objective  of  this  project  is  to  assess  the  impact 
of  this  instability  on  team  performance. 

2 

Develop  Standardized,  Representative  Tasks  for  C 
Teams  to  be  Used  in  Experimental  Studies  (1 ) 

2 

The  objective  is  to  develop  standardized  tasks  for  representative  C  team 
functions  to  be  used  in  experimental  studies  of  team  processes. 

2 

Develop  Decision  Aids  for  C  Teams  (1 ) 

The  objective  is  to  provide  improved,  interactive  decision  support  systems 
in  man-machine  interfaces  to  facilitate  team  performance.  This  project 
is  aimed  at  effective  utilization  of  the  increasing  levels  of  automation  in 
system  design. 

Assess  Applicability  of  Existing  Taxonomies  for  Learning 
Types  and  Instructional  Strategies  to  Design  of  C^TZ  (9) 


Assess  Usefulness  of  Available  Techniques  for  Sequencing 
Instructional  Content  for  C^T^  (9) 

The  overall  objective  of  these  projects  is  to  develop  an  instructional 
2  2 

methodology  for  C  T  .  Existing  methods  were  developed  in  the  context 

2  2 

of  individual  training  and  may  be  applicable  to  C  T  if  the  job  tasks  in 
2 

C  t^ams  can  be  analyzed  to  the  necessary  level. 
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2  2 

Develop  a  Problem  Space  of  Calibrated  Exercises  for  C  T  (9,  4) 

This  project  has  two  objectives:  1 )  provide  link  between  training  content 
and  job  tasks  by  means  of  a  sequence  of  training  objectives  that  progressively 
approach  job  tasks;  and  2)  provide  sequences  of  training  exercises  increasing 
in  complexity  and  difficulty  and  progressing  through  higher  levels  of  skill 
and  knowledge. 

2 

Compare  C  Job  Context  vs  Neutral  Context  for  Training  Team  Skills  (5) 

The  objective  is  to  compare  two  approaches  to  training  team  skills.  One 
approach  consists  of  training  them  in  the  context  of  the  job  tasks  where  they 
occur;  the  other  approach  consists  of  training  them  in  a  context  free  of 
job  context,  such  as  multi-person  research  games,  so  that  the  team  skills 
can  be  more  clearly  highlighted.  The  approaches  may  differ  in  ease  of 
learning  and  amount  of  transfer  of  training. 

Compare  Cost-Effectiveness  of  Live  and  Simulated 
Events  for  C^T^  Exercises  (15)  "" 

The  objective  is  to  develop  tradeoff  data  for  cost-effectiveness  evaluation 

2  2 

of  simulated  events  in  C  T  exercises.  The  evaluation  can  be  used  to 
develop  a  strategy  for  using  simulation  to  reduce  costs  and  increase  the 
training  effectiveness  of  exercises. 


124 


Develop  Type  A  System  Specification  for 
a  C'^T2  Simulation  Facility  (7) 

The  purpose  of  this  project  is  to  develop  the  functional  requirements  and 

2  2 

system  concept  for  a  simulation  facility  for  training  and  research  in  C  T  . 
The  requirements  are  derived  from  an  analysis  of  the  intended  application 
and  the  capabilities  needed  to  operate  and  support  the  facility. 

2  2 

Develop  Simulation  and  Training  Requirements  for  ECM  in  C  T  (7) 

Develop  Simulation  and  Training  Requirements 
for  Sensor  Management  in  C2t^  Exercises  ( 7 7 

These  two  projects  are  aimed  at  determining  the  requirements  to  add  these 
capabilities  to  training  exercises  to  improve  the  realism,  fidelity,  and 
effectiveness.  These  areas  are  significant  deficiences  in  current  opera¬ 
tional  training. 

Determine  Feasibility  of  Developing  Selection 
Criteria  for  Weapons  Directors  (8) 

There  are  no  selection  criteria  for  weapons  director  at  the  present  time, 
and  the  lack  of  appropriate  aptitudes  may  be  a  source  of  inadequate 
performance  in  weapons  directors.  The  objective  of  this  effort  is  to 
evaluate  the  feasibility  of  identifying  and  setting  entry  standards  for  the 
career  field  and  thereby  improving  operational  performance. 

These  recommended  projects  must  now  be  evaluated  and  assigned  a  rank 
or  priority  for  implementation.  Criteria  will  be  defined  and  applied  to 
these  recommendations  in  Chapter  IV. 
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CHAPTER  IV 


EVALUATION  OF  THE  RECOMMENDED  RESEARCH  PROJECTS 


The  16  recommended  research  projects  were  evaluated  for  priority.  The 
priority  is  expressed  as  a  rank. 

THE  EVALUATION  PROCEDURE 

Five  criteria  were  defined  against  which  to  evaluate  the  projects.  They 
are: 

•  Feasibility  of  accomplishment 

•  Utility 

•  Usability 

•  Probability  of  success 

•  Practical  payoff 

The  attributes  defining  each  of  the  criteria  are  summarized  in  Table  8. 
The  criteria  are  not  independent.  However,  they  were  used  primarily 
as  a  guideline  to  the  factors  to  consider. 

The  evaluations  were  done  by  one  evaluator  and  reviewed  by  two  human 
factors  specialists.  The  ratings  are  summarized  in  Table  9. 
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TABLE  8.  CRITERIA  FOR  EVALUATION  OF 

RECOMMENDED  RESEARCH  PROJECTS 


Technical  Feasibility  of  Accomplishment: 

--Availability  of  methods  and  data  used  for  the  effort 
--Time  to  accomplish  with  nearer-term  accomplishment  being 
more  feasible  than  longer-term  accomplishment 
--Difficulty  and  complexity  of  the  problem 
--Availability  of  specialized  resources  or  facilities  required 
--Availability  of  special  or  highly  skilled  personnel  required 
--Cost 

Utility: 

2 

--Criticality  of  C  team  functions  affected  by  the  deficiency 
--Frequency  of  the  C^  team  functions  affected  by  the  deficiency 
--Technical  leverage  acquired 

Usability: 

--Ease  of  use  by  personnel  in  training  or  operating  command 
and  research  community 

--Specialized  or  not  commonly  available  skill  and  knowledge 
required  for  use 
--Ease  of  implementation 
--Cost  to  implement  and  support 
--Ease  of  transferring  technology  to  the  user 

Probability  of  Success: 

--Subjective  estimate  of  achieving  the  objective 
Practical  Payoff: 

--Value  of  the  expected  outcome 

--Expected  improvement  in  team  performance  and  effectiveness 
--Expected  improvement  in  training  effectiveness 
--Cost  reduction 


TABLE  9.  SUMMARY  OF  EVALUATION  AND  RANKING 
OF  RECOMMENDED  RESEARCH  PROJECTS 
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“ 
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M 

H 

H 

M 
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M 
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H 
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6 
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H 

H 

M 
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learning  types  and  instructional  strategies  to 
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H 

M 

H 
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H 

M 

3 

Assess  usefulness  of  available  techniques  for 

sequencing  instructional  content  for  C2T2. 

H 

M 

H 
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H 

H 

3 

Compare  C2  job  context  vs  neutral  context  for 

training  team  skills. 

M 

M 

M 

H-M 

M 

n 

Compare  cost-effectiveness  of  live  and  simulated 

events  for  C2T2  exercises. 

M 

M 

H 

H 

M 

H 

6 

Develop  simulation  and  training  requirements  for 

ECM  in  CST2. 

L 

M 

H 

M 

L-M 

H 

H 

10 

Develop  simulation  and  training  requirements  for 

sensor  management  in  C2T2  exercises. 

L-M 

M 

H 

M 

L-M 

H 

H 

8 

Develop  Type  A  System  Specification  for  a 

C2T2  simulation  facility. 

H 

H 

H 

H 

H 

1 

Determine  feasibility  of  developing  selection 

criteria  for  weapons  directors. 

H 

H 

H 

H 

H 

1 

The  procedure  for  evaluation  consisted  of  rating  each  project  as  high, 
medium,  or  low  on  each  criterion.  Separate  ratings  for  research  and 
training  were  made  on  utility,  usability,  and  practical  payoff. 

Some  of  the  projects  were  grouped  into  the  categories  of  information 
generation,  application  tools,  and  research  tools  which  follow. 

Information  Generation 

2 

•  Field  study  for  further  definition  of  deficiencies  in  C  team 
performance  and  training. 

2 

•  Compare  "soft"  vs  "hard"  C  teams  in  performance  of  represent¬ 
ative  tasks. 

2 

•  Compare  C  job  context  vs  neutral  context  for  training  team  skills. 

•  Compare  cost-effectiveness  of  live  and  simulated  events  for 

2  2 

C  T  exercises. 

Application  Tools 

•  Develop  procedures  for  system  and  task  analysis,  compatible 

with  MIL-H-46855  and  ISD,  to  be  used  in  design  of  man-machine 
2 

components  for  C  teams. 

2 

•  Develop  decision  aids  for  C  teams. 

•  Assess  applicability  of  existing  taxonomies  for  learning  types 

2  2 

and  instructional  strategies  to  design  of  C  T  . 

•  Assess  usefulness  of  available  techniques  for  sequencing 

2  2 

instructional  content  for  C  T  . 

2  2 

•  Develop  simulation  and  training  requirements  for  ECM  in  C  T  . 
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•  Develop  simulation  and  training  requirements  for  sensor 

2  2 

management  in  C  T  exercises. 

•  Determine  feasibility  of  developing  selection  criteria  for  weapons 
directors. 


Research  Tools 


2 

•  Develop  standardized,  representative  tasks  for  C  teams  to  be 
used  in  experimental  studies. 

2  2 

•  Develop  Type  A  System  Specification  for  a  C  T  simulation 
facility. 


The  procedure  for  evaluation  in  these  categories  was: 


Evaluation  Rules 

Rationale 

Information 

generation 

Not  evaluated  on 
usability 

Usability  deals  with 
implementation  of  tool  or 
solution  and  does  not  differen¬ 
tiate  in  this  category. 

Not  evaluated  on 
practical  payoff  for 
research 

Cannot  differentiate  since 

all  information  gives  research 

leverage. 

Application 

tools 

Not  evaluated  for 
research,  except  on 
utility 

Primary  relevance  for  research 
is  technical  leverage  from 
knowledge  gained. 

Research 

tools 

Not  evaluated  for 
training 

Primary  benefit  is  for  research 
capability;  only  incidental 
direct  benefit  to  application. 
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When  each  project  was  evaluated  on  the  relevant  criteria,  a  rank  score  was 
derived  for  each  project.  Points  were  assigned  to  the  H,  M,  and  L  ratings, 
and  a  mean  of  unweighed  values  was  computed.  All  ties  were  assigned  the 
same  rank,  and  the  project  with  the  next  highest  score  was  assigned  the 
next  rank  number.  For  example,  three  projects  tied  with  the  highest 
mean  rating,  and  each  was  assigned  rank  1;  the  project  with  the  next 
highest  mean  rating  was  assigned  rank  2.  The  16  research  projects  were 
assigned  11  ranks. 

RESULTS  OF  THE  EVALUATION 

The  highest  ranked  projects  were: 

2 

•  Develop  standardized,  representative  tasks  for  C  teams  to  be 
used  in  experimental  studies. 

2  2 

•  Develop  Type  A  System  Specification  for  a  C  T  simulation 
facility. 

•  Determine  feasibility  of  developing  selection  criteria  for 
weapons  directors. 

Two  are  research  tools  which  are  basic  to  providing  facilities  to  support 

2  2 

a  research  program  in  C  T  .  The  third  project  deals  with  selection 
criteria  for  weapons  directors.  The  feasibility  study  is  well  within  the 
state  of  the  art  and  has  a  high  potential  payoff  for  training  in  the  sense  of 
increasing  the  match  between  job  requirements  and  performed  qualifications. 
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2 

4.  Field  study  for  further  definition  of  deficiencies  in  C  team 
performance  and  training. 

5.  Assess  applicability  of  existing  taxonomies  for  learning  types 

2  2 

and  instructional  strategies  to  design  of  C  T  . 

6.  Assess  usefulness  of  available  techniques  for  sequencing 

2  2 

instructional  content  for  C  T  . 

Two  additional  projects  were  recommended.  They  are  projects  with 
some  technical  risk  but  their  pervasiveness  in  all  research  areas 
warrants  their  pursuit.  They  are: 

2 

•  Construct  a  model  for  C  team  performance  in  a  CRC,  AWACS, 
or  TACC  type  of  organization  and  develop  procedures  for  system 
and  task  analysis. 

2 

•  Develop  a  performance  measurement  system  for  C  teams 
which  assesses  team  competence  effect  in  system  effectiveness 
and  needs  for  remedial  training. 
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The  three  lowest  ranked  projects  were: 

•  Develop  procedures  for  system  and  task  analysis,  compatible 

with  MIL-H-46855  and  ISD,  to  be  used  in  design  of  man- 

2 

machine  components  for  C  teams. 

2  2 

•  Develop  simulation  and  training  requirements  for  ECM  in  C  T  . 

2 

•  Compare  C  job  context  vs  neutral  context  for  training  team  skills. 
Develop  Procedures  for  System  and  Task  Analysis 

This  project  was  ranked  M  or  lower  on  technical  feasibility  and  probability 

of  success  because  the  research  problem  is  complex  and  unstructured  in 
2 

the  areas  of  C  team  behaviors.  Thus,  the  nature  and  availability  of  needed 
data  are  uncertain. 

2  2 

Develop  Simulation  and  Training  Requirements  for  ECM  in  C  T 

This  project  was  disabled  primarily  by  the  L  rating  on  technical  feasibility 
compared  with  the  companion  project  on  simulation  of  sensor  management; 
the  latter  project  was  judged  to  be  somewhat  more  tractable  a  problem. 
Simulating  ECM  will  require  an  engineering/human  factors  analysis  to 
derive  functional  implications  for  individual  and  team  performance, 
identification  of  the  fidelity  requirements,  and  task  analysis  of  the  counter- 
countermeasures.  The  state  of  this  knowledge  is  immature  and  therefore 
a  very  risky,  key  element  in  the  analysis. 
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2 

Compare  C  Job  Context  vs  Neutral  Context  for  Training  Team  Skills 

This  project  was  basically  a  moderately  complex  experimental  problem  that 
was  more  basic  than  applied  research,  and  thus,  the  ratings  fall  short 
across  the  criteria. 

2 

Construct  a  Model  for  C  Team  Performance 


This  project  received  a  rank  of  4  because  it  was  judged  M  in  technical 

feasibility  and  probability  of  success.  The  probability  was  influenced 

largely  by  the  feasibility  and  is  redundant.  It  is  a  difficult  and  complex 

problem  area,  especially  since  its  present  state  is  still  at  the  level  of 

formulating  the  research  problem.  It  is  basic  research  in  many  ways. 

However,  it  is  well  within  the  state  of  the  art  in  modeling.  A  potential 

2  2 

difficulty  is  that  the  C  T  application  may  be  large  enough  to  approach 
the  limits  of  computing  power.  However,  this  difficulty  can  be  circum¬ 
vented  by  partitioning  the  model  into  more  computable  segments  and 
reducing  the  size  by  establishing  priorities  for  features  to  be  modeled. 

The  modeling  project  is  basic  to  the  research  program  since  it  provides 

2  2 

the  conceptual  framework  and  terminology  for  C  T  .  It  will  also  produce 
useful  products  continuously  through  the  research  program,  and  parts  of 
it  will  be  incorporated  into  other  projects  in  research  and  application. 

2 

Therefore,  it  is  recommended  that  C  team  modeling  be  given  a  higher 
priority  on  the  grounds  that  the  evaluation  was  not  an  adequate  assessment 
of  its  worth.  It  will  have  highly  significant  benefit  to  the  research  program 
even  if  it  falls  short  of  success  against  the  criteria  of  the  technology  of 
modeling. 
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2 

Develop  a  Measurement  System  for  C  Teams 

This  was  also  a  significant  project  whose  ranking  (5)  was  lowered  because 

its  complexity  and  difficulty  resulted  in  M  ratings  on  technical  feasibility 

and  probability  of  success.  Even  if  the  total  objective  is  not  attainable, 

2  2 

significant  improvements  in  C  T  can  be  achieved  from  partial  products 
and  methods  produced. 

The  two  projects  for  assessing  the  applicability  and  usefulness  of  existing 

knowledge  and  techniques  for  instructional  strategies  and  sequencing  of 

instructional  content  received  moderately  high  ratings.  They  are  relatively 

2 

easy  to  do  and  inexpensive,  however,  and  will  contribute  to  the  C  team 
model.  They  should  be  considered  for  implementation. 

RECOMMENDED  CANDIDATES  FOR  IMPLEMENTATION 

The  16  projects  were  divided  into  thirds  as  an  aid  in  selecting  candidates 
for  implementation.  The  top  third  (six  projects)  are  recommended  as 
candidates.  They  are: 

2 

1.  Develop  standardized,  representative  tasks  for  C  teams  to  be 
used  in  experimental  studies. 

2  2 

2.  Develop  Type  A  System  Specification  for  C  T  simulation  facility. 

3.  Determine  feasibility  of  developing  selection  criteria  for 
weapons  directors. 

2 

4.  Field  study  for  further  definition  of  deficiencies  in  C  team 
performance  and  training. 
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5.  Assess  applicability  of  existing  taxonomies  for  learning  types 

2  2 

and  instructional  strategies  to  design  of  C  T  . 

6.  Assess  usefulness  of  available  techniques  for  sequencing 

2  2 

instructional  content  for  C  T  . 

In  addition,  the  following  should  be  candidates  for  the  reasons  discussed 
above: 

2 

•  Construct  a  model  for  C  team  performance  in  a  CRC,  AWACS, 
or  TACC  type  of  organization  and  develop  procedures  for  system 
and  task  analysis. 

2 

•  Develop  a  performance  measurement  system  for  C  teams  which 
assesses  team  competence  effect  in  system  effectiveness  and 
needs  for  remedial  training. 

A  final  comment  is  that  one  should  consider  modifying  the  criteria  for 
evaluation  when  making  one's  own  ratings.  For  example,  the  criteria 
might  be  weighed  differentially  rather  than  equally.  Further,  some 
improvements  can  be  made  in  the  criteria.  The  interdependence  among 
them  can  be  reduced.  Probability  of  success,  for  example,  is  heavily 
influenced  by  technical  feasibility  ana  thus  highly  redundant.  Practical 
payoff  should  be  better  defined  so  that  degrees  of  payoff  are  more  readily 
differentiated. 

One  approach  to  be  considered  is  to  conceptualize  the  evaluation  in  terms 
of  probability  of  success  and  value.  Probability  of  success  would  be  a 
function  of  factors  such  as  complexity,  present  state  of  formulation  of  the 
problem,  maturity  of  the  needed  technology,  availability  of  resources. 
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time  period  for  accomplishment,  and  cost.  Value  might  be  defined  in 

terms  of  utility,  usability,  and  anticipated  improvement  in  the  cost- 
2  2 

effectiveness  of  C  teams  or  C  systems. 

This  approach  can  easily  become  more  complicated  and  expensive  than  the 
input  data  warrants.  However,  the  criteria  for  evaluation  are  important 
for  planning.  The  criteria  used  in  this  analysis  were  an  approximation 
that  was  attainable  within  the  time  and  resources  available.  Possible 
improvements  are  apparent  in  the  hindsight  following  an  attempt  to  apply 
them. 

SUMMARY 

The  16  recommended  research  projects  were  evaluated  against  the  criteria 
of  technical  feasibility,  utility,  usability,  probability  of  success,  and 
practical  payoff.  The  projects  were  ranked  on  the  basis  of  ratings  on 
these  criteria. 

The  top-ranked  third  of  the  projects  were  recommended  as  candidates  for 
evaluation.  They  are: 

2 

1.  Develop  standardized,  representative  tasks  for  C  teams  to 
be  used  in  experimental  studies. 

2  2 

2.  Develop  Type  A  System  Specification  for  C  T  simulation 
facility. 

3.  Determine  feasibility  of  developing  selection  criteria  for 
weapons  directors. 
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